GSoc 2012 Final Report

I had a great summer engage in a project for Banshee media player developing a voice control extension for it. My main goal was to enhance the usability and accessibility of Banshee.
For that I used the pocketsphinx and sphinxbase packages of CMUsphinx speech recognition tool kit. It was challenging to integrate it to Banshee media player as it has used Mono technologies and CMUsphinx is developed using C. I used the gstreamer pipeline to get the input continuously from the microphone. To enable the extension first PocketSphinx has to be added to your system.
These are the steps you should follow in Ubuntu.

First download pocketsphinx and sphinxbase packages and wnzip them.

Then build them and installed them

% ./configure
% make
% make install

do this for both folders. 

Then ran the 

export LD_LIBRARY_PATH=/usr/local/lib
export PKG_CONFIG_PATH=/usr/local/

lib/pkgconfig

command in the terminal.

then pocketsphinx_continuous command worked.

please refer to this link for more information 

http://cmusphinx.sourceforge.

net/wiki/tutorialpocketsphinx

This branch contains the voice control extension developed.

https://gitorious.org/banshee-community-extensions/banshee-community-extensions/trees/voicecontrol

clone the branch and run make and make run commands to build and install the extensions in the repository. 

I have included the language model file in a folder inside the voice control extension. 

7552.lm file is the language model file. 

when I build and install the extension the pocketsphinx will continuously convert the voice into text. 

So I called the methods to  do the basic control of Banshee according to the voice commands.

It worked for "PLAY" , "PAUSE " and "STOP" you can try it. 

So I tested the implemented code for a long time and found some times Banshee get crashed. 

It was difficult to reproduce the bug also.

this the log I got when it got crashed,

Native stacktrace:

banshee() [0x80e126c]

banshee() [0x812046c]

banshee() [0x805fe7d]

[0x4002040c]

[0x40e72d88]

/usr/lib/i386-linux-gnu/

libgobject-2.0.so.0(g_closure_

invoke+0x1ac) [0x4169dc4c]

Debug info from gdb:

Could not attach to process.  If your uid matches the uid of the target

process, check the setting of /proc/sys/kernel/yama/ptrace_

scope, or try

again as the root user.  For more details, see /etc/sysctl.d/10-ptrace.conf

ptrace: Operation not permitted.

No threads.

==============================

==============================

=====

Got a SIGSEGV while executing native code. This usually indicates

a fatal error in the mono runtime or one of the native libraries 

used by your application.

==============================

==============================

=====

I was spending much time trying to fix the bug above mentioned. But it was not resolved.

I tried these options to overcome it.

It also might be that an exception is being thrown in the managed (C#) callback that's being invoked by C code. When managed exceptions propagate up the stack, they cannot pass though C frames, and cause
crashes. I put a try/catch inside the lambdas that passing to the C code.

But I got the same scenarios of crashing Banshee unexpectedly. 

and even It looks like Mono can't use the native debugger (gdb) to backtrace the process when it crashes because it doesn't have permissions to attach. 

It was very challenging to manage all the work during the academic times. I got the chance to participate in two summits because of GNOME , Banshee , Mono ,Google and other open source based giants which were held in Boston and Spain. I always tried to contribute as much as possible to the community. 

There were no proper documentation for Banshee which I can refer. So I have to go through the source code and identify the Banshee methods.

The above mentioned bug was a bottle neck for my progress of the project. If I got through that problem within the project period I could have done a better job. 

I want to thank the GNOME & Banshee community for helping students to get involved with free software development, Google for giving me the opportunity of work with GNOME & Banshee.

I'm hoping to continue contributing to GNOME & Banshee. 

There are lot of future work I have planned. I hope I will have much time engage in that kind of things. 

I learned so many new things and was able to make a professional network by doing this project.
I was able to improve my technological   knowledge and also soft skills by engage in work for GNOME.