Peter's Devlog

Using the Asterisk Voip PBX as SIP Mediaserver

2008-04-25T09:39:00.000-07:00

Here I show how the open source Voip PBX Asterisk can be used as a simple mediaserver for Sip applications.

Prerequisites

I use the AsteriskNOW software appliance which is prepackaged ready-to-run version of Asterisk. It can be downloaded from http://www.asterisknow.org/downloads . There is a very good documentation avaliable as PDF download from Asterisk Book Download .
Asterisk is running on Linux only, so in order to run it on Windows or other OS you need a virtual machine environment e.g. VMware. The download from AsteriskNOW is an ISO image, to run it on VMWare you need VMWare Server which you can get for free from http://www.vmware.com/download/server/ . Alternatively you can use VMware Player and run an Asterisk Virtual Appliance which you can get from the VMware virtual appliance directory at http://www.vmware.com/appliances/ .
I prefer to run it from VMware Server. To install it create a new virtual machine, insert a CD made from the ISO image or mount the image directly and follow the instructions of the graphic installer. There is quickstart guide available at http://www.asterisknow.org/support/install .
Asterisk provides a Web based administration, but for our usecases it is required to have direct access to the Asterisk console. Use an ssh client (e.g. Putty from http://www.chiark.greenend.org.uk/~sgtatham/putty/ ) to connect to the Asterisk server.

Configuration

Connect to Asterisk using the ssh client and login (default user is admin and password is password). Configuration of Asterisk is done by editing various configuration files. When you edit the configuration files you must give yourself the access rights with sudo, e.g sudo vi sip.conf.
The sip configuration file is sip.conf in /etc/asterisk. Rename the file from the installation (it is very complicated and contains lots of examples) and create a new sip.conf as follows:

[general]
context=default
allowoverlap=no
bindport=5060
bindaddr=0.0.0.0
srvlookup=yes
[sailfin]
type=peer
context=from-fwd
host=192.168.0.12
insecure=port

The configuration is done by declaring contexts, in our case the sailfin context, type=peer means that this context can only receive calls.The host address has to be replaced with the ip address of the sailfin server (yes the sailfin server since it is the peer, not the Asterisk server). Detailed explanation about contents can be found in the Asterisk book.
The next configuration file is extensions.conf, which contains the dialplan. Rename the file and create a new one like for sip.conf. A dialplan tells Asterisk what to do when a call is received. In our example we are using a very simple dialplan which tells Asterisk to play an announcement when a callis received on a certain extension.

[from-fwd]
exten => 1234,1,Answer()
exten => 1234,2,Playback(to-reach-operator)
exten => 1234,3,Hangup()

When a call is received for the context from-fwd with the extension 1234 the call is answered, then an announcement is played and the call is finished. Announcement are defined as .wav files with a full path, if no path is specified the default directory /var/lib/asterisk/sounds is used.
To activate the context type sip reload from the console (the console can be reached from the Asterisk main menu with ALT-F9). There are many other sip related console commands, they are explained in detail in the Asterisk book in appendix E.
The log files for sip can be found in /var/log/asterisk in file full, errors are in file messages.
When you now make a sip call to Asterisk (sip:1234@192.168.0.12 in our example) Asterisk answers the call with 200 OK, then receives an ACK, plays the announcement and ends the call with BYE when the announcement is finished.

Next Steps

The usecase described here is an extremely simple one, Asterisk can do much more than that, e.g. it can act as

IVR with voice menus reacting on keypresses and event database access.
Voice mail server
Call recording server

I will try to explore these possibilities and explain in a future post.

Hunting Sample App using ECharts

2008-04-24T12:55:00.000-07:00

In this example I show how to build a more complex sample using the ECharts state machine language. Details about ECharts can be found in http://echarts.org/ and also in one my previous posts in screening sample app with ECharts . The sources for the sample are available in cvs using cvs -d:pserver:[your java.net account]@cvs.dev.java.net:/cvs co sailfin/sailfin-tests and then can be found then in community/samples/hunting, or send an email to me to get them.

It provides the functionality to define a list of alternative Sip URI's in a database which are tried in sequence to connect when a call is received until a successful connection is established.

The application is triggered only in terminating case via the ECharts application router if the To address matches the specfied pattern in the app router.
It then connects the call to a mediaserver (MRF) which plays a dialtone or other announcement to the caller.
When the dialtone is established it sends a INVITE to the original called party of the call.
When the called party answers with 200 OK it disconnects from the mediaserver and reconnects via REINVITE to the caller. How to setup a mediaserver I will show in a future blog entry.
If the called party returns a 486 BUSY or 487 NOT REACHABLE it retrieves the next alternative Sip URI from the database and sends INVITE again.
If no alternative called party is available it stops the dialtone and signals a failure to the caller.

Sip Callflow for Redirect on Busy

The following callflow shows a scenario where the B Party is busy and the call is forwarded to C-Party.

A-Party......Sailfin......B-Party......MRF......C-Party

-INVITE SDPa->.....................................
.............-INVITE-------------------->..........
.............<-200 OK SDPm---------------..........
<-200 OK SDPm-.....................................
-ACK--------->.....................................
.............-ACK----------------------->..........
.............-INVITE SDPa--->......................
.............<-486 BUSY------......................
.............-ACK----------->......................
.............-INVITE SDPa------------------------->
.............<-200 OK SDPc-------------------------
.............-BYE----------------------->..........
<-200 OK SDPc-.....................................
.............<-200 OK--------------------..........
-ACK--------->.....................................
.............-ACK--------------------------------->
..................t a l k i n g....................
-BYE--------->.....................................
.............-BYE--------------------------------->
.............<-200 OK------------------------------
<-200 OK------.....................................

State Machines

A graphical description of the state machines can be found in the downloaded sources in hunting/src/echarts/hunting/ech/doc-files

The MainFSM machine connects to the MRF to play the dialtone using the ConnectFSM machine. When the dialtone is established it transitions to the HuntingFSM which performs the search and connect for the called party. Finally when the call is successfully established it transitions to the TransparentFSM which handles the talking state of the call and also the turndown at the end.
The HuntingFSM runs three paralles states.
The PLAY_DIALTONE state uses the TransparentFSM and monitors the connection to the MRF.
The CALL state uses the CallFSM to make a connection the called party until it has a successful connection.
The REINVITE state uses the ReinviteFSM to handle the reinvite to the caller after a successful connection to the B-Party has been established.
The ConnectFSM machine handles the conenction to the MRF.
The CallFSM handles connect attempts to the B-Party.
The SendReinviteFSM handles the reinvite to the caller.
The TransparentFSM handles the talking state and turndown together with the TransparentHandleRequestFSM.

Screening App uses ECharts Application Router

2008-03-01T12:14:00.000-08:00

The screening application now uses the ECharts application router. A good description of the application router can be found in http://echarts.org/ECharts-for-SIP-Servlets/JSR116-Application-Router-Deployment.html.

In the moment the screening app uses the application router only to control triggering so that it is only triggered in originating but not in terminating calls. In the future different kinds of screening could be applied to originating or terminating calls or the screening app could be integrated with other applications in application composition.

Following changes were made to the screening app:

A new servlet "InitialServlet" was added.This servlet is triggered when the application is not determined yet and handles the initial routing.
In sip.xml the InitialServlet was added and the mapping rules were enhanced to use the results of application routing.
The ConnectFSM was enhanced to call the approuter before it sends out an initial INVITE.
The approuter.xml file was created, this file has to be copied to sailfin/domains/domain1/config.

Screening Sample App using ECharts (Part 2 Web Admin)

2008-02-24T08:34:00.000-08:00

The third parts shows how to build a web based administration for the screening application. It is built using the Visual Web JavaServer Faces framework. It provides basic CRUD (Create, Read, Update, Delete) functionality for each of the entities in the database.

Database Contents

The database contains the following entities:

UserData:
holds the user id, the name of the used screening profile, the reference to the provider data and the users's Sip Uri.
ProviderData
holds the system specific data, in our case the Sip Uri of the announcement to be played when a call is not allowed.
ScreeningProfile
holds the screening data to be checked in blacklists and whitlelists. Each entry consists of a profile name, a blacklist/whitelist indicator and the Sip Uri to be checked itself.

Page Creation

The application consists of a main page which has links to the Users page, the Providers page and the Screening page. The application is modeled according to the tutorial "Modifying Database Table Rows with the Java Persistence API" in http://www.netbeans.org/kb/60/web/web-jpa-part2.html. Do the following steps to build the application:

Right click on the screening application in the project window and select "New-Visual Web JSF Page" for each page.
Right click on the screening application in the project window and select "New-Visual Web JSF Session Bean" once.
Drag and Drop the hyperlinks for users, providers and screening to the main page.
Add the code for three properties userDataList, providerDataList and screeningProfileList to SessionBean1 as well as the corresponding methods to fill this lists from the database (updateUserData(), updateProviderData() and updateScreeningProfile() ) and one helper method findProviderDataById(String providerId).

Page Layout, each entity page contains:

a static text field for the headline
a grid panel with id mainContainer
a table which is bound to the data in SessionBean1
a grid panel with id buttonPanel with 2 buttons, an add-update button and a delete button
a grid panel with name addUpdatePanel with one label and one text field or dropdown list for each field in the entity.
a hyperlink to go back to the main page

Page Flow, open the faces-config.xml file and in the graphical editor

Connect each of the hyperlinks on the main page with the corresponding entity page
Connect the backlink on the entity pages with the main page
Connect the buttons on each entity page with the entity page itself

Binding to Data

Right click the table and select "Table Layout", then select the corresponding list in "Get Data From".Set the id's of the table columns as [property name]Column and the table fields as [property name]Text.
Set the id's of the addUpdatePanel elements as [property name][field or list]
.For the dropdown list for providers in the Users page select it, right click and select "Bind to Data", then choose providerIdList.
For the dropdown list for screening profiles in the Users page select it, right click and select "Bind to Data", then choose screeningNameList

Handling Selection using an addtional radio button in each table row

Add the code for a TableSelectPhaseListener to the java bean for the page as shown in the example code
Add the javascript code for initAllRows() as shown in the example code. This code is specífic for each page because it also updates the addUpdatePanel fields when a table row is selected.
Right click the table and select "Table Layout", then "New" and select Radio Button from component type and click "Up" until the button is on the top of the column. Set the button's id to selectButton.
Select the radio button, right click in the navigator window and select "Property Bindings". Add the bindings for selected, selected value and name. For name enter manually
"#{[page name]selectButton.id}". This binds the select button to the TableSelectPhaseListener object.
Select the selectColumn in the navigator window and set the bindings for selectId to
"#{[page name]selectButton.id}" and the bindings for onClick to "setTimeout('initAllRows()', 0)".This performs the update of the table when a radio button is clicked.
Select the tableRowGroup in the navigator window and set the bindings for selected to the selectedState property.

Command Handlers

Add persistency to the page by right clicking in the source editor and then select"Persistence->Use Entity Manager".
Add the code for merge() and delete() as shown in the example code. You can delete the generated code for persist().
Doubleclick the "add-update" button and enter the java code as shown in the example code.
Doubleclick the "delete" button and enter the java code as shown in the example code.

Build and deploy the application, then point the webbrowser to http://localhost:8080/screening

To make an initial fill of the database go to the screening directory and execute ant -f tests.xml filldata

Screening Sample App - ECharts Details

2008-02-08T23:44:00.000-08:00

The following diagram show the structure of the screening application, the diagram is generated by the build process in the echarts/screening/ech/doc-files folder. From the initial state RECEIVE_INVITE either the state PLAY_ANNO is reached (if screening is applied) or the state CONNECT if screening is not applied. When the connection is successfully established the state TALKING becomes active which handles the call until finish.

Screening Sample App using ECharts (Part 1 - Application)

2008-02-04T12:49:00.000-08:00

In this example I will show how to build a simple screening application for Sailfin using the ECharts state machine language. The application checks the dialed number and either let the call proceed or plays an announcement to the caller that the call is forbidden. Each user is assigned a screening profile which contains his screening rules consisting of blacklisted numbers and exceptional whitelisted numbers. A call is forbidden if the dialed number is contained in the blacklist but not in the whitelist. A call is allowed if the dialed number is not contained in any list or in the whitelist only.

What is ECharts

According to the description on the ECharts website (http://echarts.org/) it is a state machine-based programming language for event-driven systems derived from the standardized UML stacharts language. ECharts is a hosted language which means that it is dependent on an underlying programming language such as Java.

Benefits of ECharts

It provides a problem oriented language. State machines are a typical method to describe message processing applications and a language whose paradigm fits to the domain makes application development easier.
From an application developers point of view your application has to handle only a single call in contrast to the servlet programming model where you have to deal with all calls handled by the servlet in parallel. The session and dialog management is abstracted by the so called box model.
It allows to build reuseable parts by defining state machines which handle a certain task and can be inserted as a single state in an outer state machine. In the example I will show a Connect machine which builds up a connection to the called party, a Transparent machine which handles the talking state and a PlayAnnouncement machine which plays announcements to the caller.

Prerequisites

Get the ECharts 2.2. SDK from http://echarts.org/Downloads.html and unzip it.
The ECharts compiler uses Python, if do not have it get it from http://www.python.org/download/
If on order to use the documentation generator for state machines on Windows you need graphviz from http://www.graphviz.org/. For some strange reasons I had to directly set the path for the graphviz "dot" program in ech2javadoc to make it working.
In the sailfin application server set the following JVM options with the admin console on http://localhost:4848/ by "Application Server->JVM Settings->JVM Options->Add Option".
-Dorg.echarts.servlet.sip.messagelog=true
-Dorg.echarts.debugging=true
-Dorg.echarts.servlet.sip.debugging=true
-Dorg.echarts.servlet.sip.logdir=${com.sun.aas.instanceRoot}/echlogs
-Dorg.echarts.system.transitionTimerManager.class=
org.echarts.servlet.sip.TransitionTimerManager
The sources are available in cvs using
cvs -d:pserver:[your java.net account]@cvs.dev.java.net:/cvs co sailfin/sailfin-tests
and then can be found then in community/samples/screening, or send an email to peter.c.klein@siemens.com to get them.

Creating the Screening Sample

Create a new database with database name "screening" and a new Converged Servlet Application with name screening like in http://petersjb.blogspot.com/2008/01/create-sip-application-for-sailfin.html
Add the entity classes for UserData, SceeningProfile and ProviderDataand copy their sources from the downloaded cvs sources.
Add the java class for ModelFacade as well as the java class for RegisterSipServlet and copy its sources from the downloaded cvs sources.
Add a java class FSMSupport which contains helper functions called by the state machines and copy its sources from the downloaded cvs sources.
Add antlr.jar, approuter.jar, echarts.jar and echarts-sipserlet.jar in the /lib folder to the projects libraries.
Create a new folder structure echarts/screening/ech in screening/src to hold the echarts sources and copy the *.ech files from the downloaded cvs sources. Add this folder in the project properties to sources.
Add to the build.xml file the properties and targets -pre-compile (for ech2java compilation) and -post-compile (for generating state machine documentation) as in the build.xml file in the downloaded sources.
Add the EChartsSipServlet and the RegisterSipServlet to the sip.xml as shown in the example code. Note that the state machines themselves are no servlets but are called from the EChartsSipServlet. Also add ModelFacade to web.xml
Compile and deploy the application.
Setup the data in the database as shown in the filldata.sql file in the
"scenarios/tc-001-basic-ua" folder.
Depending on the dialed number and contents of the screening profile the call will either be connected to the called party or the party defined in the ProviderData.anno attribute.

I the next post I will show the web based admin and some insights how it works.

Notes

The web based admin is already included in the sources but in the moment only viewing works, modify and delete have to be added.

Create a SIP Application for Sailfin (Part 3 Webapp)

2008-02-01T13:18:00.000-08:00

The third and last part shows how to build a simple web based administration for the user database with basic create, read, update and delete functionality. The application is modeled after the sample in http://www.netbeans.org/kb/60/web/inserts-updates-deletes.html.

View the Data

Create the web page with "New->Visual Web JSF Page" and a session bean with
"New->Visual Web JSF Session Bean".
Open the visual web page editor for the created page. Then go to the "Services" tab, connect to the database and select the USERDATA table. Drag and drop it to the editor's canvas. You will see in the navigator window that a Data Provider object and a Row Set object were created.
Drag and drop a table on the canvas. Select the table, right click and select "Bind to Data". Choose the userdataDataProvider which was created before and click OK. The table is now able to display data for users.
Deploy the application and point your browser to localhost:8080/b2bua to view the contents of USERDATA.

Create and Delete

Add a button "Add User" and a button "Save Changes".
Doubleclick the "Add User" button and enter the action code from the sources.
Doubleclick the "Save Changes" button and enter the action code from the sources.
Select the table and open the layout editor. Add a new column and change the type of the column to "Button".
Doubleclick the "Delete" button and enter the action code from the sources.
Deploy the application again. You can now create and delete user data. The changes are stored in the database when you click "Save Changes".

Create a SIP Application for Sailfin (Part 2 Servlets)

2008-02-01T01:43:00.000-08:00

In this part we will add a Sip Servlet for registration (handling the REGISTER message and storing the users contact information in the database) and a Sip Servlet which acts as a B2B User Agent transparently routing between A and B party.

Create Servlets

Create a new class ModelFacade in the sip.model package. This class contains the methods UserData findUserDataById(String userId) and UserData updateUserData(UserData user) which are used by the Registration Servlet.
Create a RegisterSipServlet class and a B2BUASipServlet class with "New->Sip Servlet".
The code for both you can copy from provided sources.
In configuration files add the RegisterSipServlet and the B2BUASipServlet to sip.xml and the ModelFacade to web.xml.
Deploy the application.

Prerequisites for Testing

Get a softphone (e.g. X-Lite from http://www.counterpath.com/) and configure its settings (e.g. username to "Bob", domain to "test.com", proxy to localhost:5060).
If you are running the second softphone on the same machine you need a different model as second phone (e.g. 3CX VOIP phone from http://www.3cx.de/voip-telefon/index.html) and configure it for a second user (e.g. Alice)

Test the Application

Start both softphones, on X-Lite you should see "Ready, your username is Bob" on the display
Dial from A party as sip:user@siphost:port, e.g. sip:Alice@localhost:5057
The second phone rings and you can start the conversation
When you go onhook the call is terminated.

Automatic Testing with SIPp tool

Download the SIPp tool from http://sipp.sourceforge.net/. On Windows you also have to download WinPCap from http://www.winpcap.org/devel.htm
Run a basic test with ant -f tests.xml tc-001-basic-ua

Create a SIP Application for Sailfin (Part 1 Database)

2008-01-31T10:03:00.000-08:00

I will go step by step through creating a simple SIP application using the NetBeans IDE for the Sailfin Sip application server. In part 1 we will setup the user database where the Sip users will register themselves.

Prerequisites

get the NetBeans IDE from http://www.netbeans.org/community/releases/60/index.html and the NetBeans Sip application development from https://sailfin.dev.java.net/servlets/ProjectDocumentList (in the appdevnb section).
follow the installation instructions in http://blogs.sun.com/nb_sip_sailfin_installation/resource/nb_sip_sailfin_installation.html
The sources are available in cvs using
cvs -d:pserver:[your java.net account]@cvs.dev.java.net:/cvs co sailfin/sailfin-tests and then can be found then in community/samples/b2bua, or send an email to peter.c.klein@siemens.com to get them.

Preparing the database:

Open the database settings in NetBeans with "Tools->Java DB Database->Settings" and select your Java DB installation (normally in sailfin/javadb) and the database location (normally in sailfin/databases).
Start the DB server with "Tools->Java DB Database->Start Server".
Create a new database with "Tools->Java DB Database->Create Database", then enter the database name (b2bua), the username (APP) and password (APP).
You can then connect to the database from the Servers tab. Click on the database connection, right click and then Connect.

The application itself is a simple Sip back-to-back user agent which allows to build up a communication between two Sip phones and acts as a Registrar which handles the registration of the Sip users.

Creating the Data Storage

Create a new project with "File->New Project" and choose Sip and Converged Servlet Application as project type. The project name of our sample is b2bua.
In the project view select Source Packages, right click and choose "New->Sip Servlet", enter RegisterSipServlet as class name and b2bua.sip as package name.
In the project view select Source Packages, right click and choose "New->Entity Class", enter UserData as class name and b2bua.model as package name. In the same dialog box click on "Create Persistence Unit".
Choose "New Data Source" and in the next dialog box set jdbc/b2bua as JNDI name and select the prevously created database connection to b2bua.
Look at the downloaded source code for UserData.java. You will see that the attribute sipURI and its getter and setter methods were added. In sipURI the URI of the registered user is stored.

Deployment of Part 1

Open the projects properties, go to the "Run" page, select Java System Application Server as server and uncheck the "Display Browser on Run" checkbox.
Select "Run->Run Main Project" to deploy the application. Depending on the sailfin version you are using there might be some warnings during deployment. Important is the output line "Enable of b2bua in target server completed successfully".
Check the deployment on the sailfin admin console. Point your browser to http://localhost:4848/ and check "Applications->Sip->Converged Sip Modules" entries. b2bua should be visible as deployed.
Connect to the database, open the USERDATA table, right click and select "View Data". The table should be empty. Add data by entering "insert into USERDATA values ('Bob@test.com', ''); in the SQL window, right click and select "Run Statement".
You can also another SQl client, e.q Squirrel (download from http://squirrel-sql.sourceforge.net/, good instructions can be found in http://db.apache.org/derby/integrate/SQuirreL_Derby.html). Squirrel allows to directly modify the database without using SQL statements

Welcome to my Devlog

2008-01-17T12:20:00.000-08:00

Hello,

This weblog tells you about the private development projects I am involved in, currently this are:

PinTheGlobe: a GoogleMaps based website (http://www.pintheglobe.com) which allows to view and add georeferenced information (so called geotags) to the map, e.g. you can add favorite places you want to share with others. I created this website mainly to learn new website development techniques like javascript and Ajax.

Sailfin: participating in sample applications and testing for the sailfin project. Sailfin (https://sailfin.dev.java.net/) is an open source converged SIP and HTTP application server based on glassfish.