Git & BitBucket

To use BitBucket and Git:

1. Create a repository on BitBucket first. 
2. Create a new folder of interest in your development environment. 
3. Add some files. 
4. Type "git init" to initialize the current folder as repository. 
5. Type "git add ." to add all files. 
6. Type "git commit -m "Initial."" to perform initial commit. 
7. Type "git push https://bitbucket.org/user/new_repo master" to push changes into your new BitBucket repository. 

Raspberry Pi Setup

After loading the Raspberry Pi image, the device boots into the raspi-config tool. But since the Pi is powered off a USB port off my desk (not recommended by the manual, presumably because the USB port may or may not supply enough current), I'd like to do some additional setup to facilitate my development process: 

1. Wifi setup

Wifi setup was fairly straight forward with a compatible Wifi dongle. I bought the Asus N10 from a local retailer and followed the instruction here: http://www.raspberrypi-tutorials.co.uk/set-raspberry-pi-wireless-network/

More specifically, all I had to do was: 

sudo vi /etc/wpa.config

And add: 

network={
ssid="MY_SSID"
proto=RSN
key_mgmt=WPA-PSK
pairwise=CCMP TKIP
group=CCMP TKIP
psk="WPA-PASSWORD"
}

Then: 

sudo vi /etc/network/interfaces

And add: 

auto wlan0
iface wlan0 inet dhcp
wpa-conf /etc/wpa.config

And restart the network interface: 

sudo /etc/init.d/networking restart

Now I can ssh into the Pi using Putty. 

Edit: 
I tried to set this up with using a keyboard connected, the Pi had problem reading the conf file and led to wpa_supplicant failure, even though the dongle was detected correctly (ifconfig shows wlan0). 

I think this is relating to the default keyboard layout of the Pi and the fact I'm using an international keyboard. 

I connect the Pi locally using an ethernet cable, ssh'ed into the Pi and re-wrote the config file and issued ifdown and ifup to wlan0. The interface came up correctly. 

2. Disable overscan 

When my Pi boots up there's a black margin around the console. I'm not sure why this happens, but all I had to do was to disable overscan: 

sudo vi /boot/config.txt

Uncomment the following line: 

disable_overscan=1

Power cycle the Pi. 

3. Extend the partition

By default the image has a partition of < 60M. To extend that: 

sudo raspi-config

And select: 

expand_rootfs

Wait and reboot. Note that the resizing on the next boot will take a bit of time. My 16GB took about 15 minutes or more. 

4. Mount the Windows share drive: http://www.swerdna.net.au/susesambacifs.html

mount -t cifs -o username=guest,password=guest //192.168.1.100/share /mount/share

5. Install pv (file copy with progress indicator): http://www.commandlinefu.com/commands/view/5107/copy-a-file-using-pv-and-watch-its-progress

sudo apt-get install pv
pv source > destination

Free Private Online Repository

I recently discovered a free online code hosting service: Bitbucket (https://bitbucket.org/). 

Most other services I looked at previously forces the code to be open sourced if not paid. The biggest bonus for Bitbucket is that it provides free private online repository service (but only with up to a limited developers per project). 

However, for my purpose (private projects on the sides and such), it's sufficient. 

Raspberry Pi File Sharing

To perform file sharing with Raspberry Pi, I had to install the Samba server. General instruction can be found here: 
http://ubuntuforums.org/showthread.php?t=202605

I had to add a section of my smb.conf file at the end: 
sudo vi /etc/samba/smb.conf

[Share]
comment = guest access share
path = /home/pi/downloads
browseable = yes
read only = yes
guest ok = yes

And restart the Samba server. 

Now everyone can access my /home/pi/downloads directory. 

Capturing Frame Buffer

On the Raspberry Pi the default the framebuffer depth seems to defaults to 16bit (I don't know if this true with other Linux distribution). I want to capture frame buffer at 24bits per pixel:

Set the frame buffer to the right depth:
fbset -depth 24

Capture the frame
cat /dev/fb0 > screenshot.raw

For some reason the captured frame provides more data at the end. With 24 bits (3 bytes) per pixel the actual expected frame size is 1920x1080x3 = 6220800. I was able to just truncate the file and drop the rest:
truncate --size 6220800 screenshot.raw

To convert this back to a readable png format:
ffmpeg -vcodec rawvideo -f rawvideo -pix_fmt rgb24 -s 1920x1080 -i screenshot.raw -f imag2 -vcodec png screenshot.png

SSH Auto Login with Putty

One of my Raspberry Pis is powered on 24/7. To access it I always SSH into it using Putty. It took a couple weeks beforeI got fed with entering my username and password every time I want to work on it. 

I did a quick Google search and came to this: 

http://superuser.com/questions/44106/is-there-a-way-to-auto-login-in-putty-with-a-password

Which essentially come to creating a shortcut to putty.exe, and add the following argument: 

pi@192.168.1.151 -pw raspberry

Now I can access my Pi by simply double clicking on the shortcut. 

Raspberry Pi as a hardware assisted H264 encoder

One of the amazing things about the Pi is despite its limited CPU power, it has an amazing GPU. This can observed from the fact: 

• It can do high profile h264 encode/decode (1080p @ 30 fps)
• It can play Quake 3
• It can run fancy Qt demos 

Just recently the default Raspberry Pi firmware now has the h264 encoder feature. kulve from the Raspberry Pi forum gave a step by step instruction in the following post: 

http://www.raspberrypi.org/phpBB3/viewtopic.php?f=63&t=18261

I've captured the code here in case the link expires someday: 

# Get The Hobbit in 640 x 272 H264
youtube-dl -f 18 "http://www.youtube.com/watch?v=SDnYMbYB-nU"
 
# Convert 9 seconds of frames to PNG
mplayer -vo png -nosound -ss 61 -endpos 9  SDnYMbYB-nU.mp4
 
# Convert PNG frames to raw RGB (don't ming the weird .png.raw naming)
for i in *png; do convert $i rgb:$i.raw; done

hello_video.bin testout.h264

ffmpeg -f h264 -i testout.h264 -vcodec copy outtest.mp4

mplayer outtest.mp4

I had to install a couple utilities, including youtube-dl, convert, mplayer and ffmpeg. The youtube-dl from the repository seemed to be obsolete already, and I had to download a new one from the official source. 

The hello_video.bin is written by kulve from the Raspberry Pi forum, with the code listed below: 

/*
Copyright (c) 2012, Broadcom Europe Ltd
Copyright (c) 2012, Kalle Vahlman 
                    Tuomas Kulve 
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
    * Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.
    * Neither the name of the copyright holder nor the
      names of its contributors may be used to endorse or promote products
      derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

// Video deocode demo using OpenMAX IL though the ilcient helper library

#include 
#include 
#include 

#include "bcm_host.h"
#include "ilclient.h"

#define RAW_WIDTH    640
#define RAW_HEIGHT   272
#define RAW_BPP      3
#define RAW_SIZE     (RAW_WIDTH * RAW_HEIGHT * RAW_BPP)

static int read_raw_rgb(void *buf, OMX_U32 *filledLen, int filenumber)
{
  char filename[256];
  FILE *f;
  size_t ret;

  snprintf(filename, 256, "%08d.png.raw", filenumber);

  f = fopen(filename, "r");
  if (!f) {
 printf("Failed to open '%s'\n", filename);
 return 0;
  }

  ret = fread(buf, 1, RAW_SIZE, f);
  if (ret < RAW_SIZE) {
 printf("Failed to read '%s': %d\n", filename, ret);
 return 0;
  }

  *filledLen = RAW_SIZE;

  fclose(f);
  return 1;
}

static void print_def(OMX_PARAM_PORTDEFINITIONTYPE def)
{
   printf("Port %lu: %s %lu/%lu %lu %lu %s,%s,%s %lux%lu %lux%lu @%lu %u\n",
          def.nPortIndex,
          def.eDir == OMX_DirInput ? "in" : "out",
          def.nBufferCountActual,
          def.nBufferCountMin,
          def.nBufferSize,
          def.nBufferAlignment,
          def.bEnabled ? "enabled" : "disabled",
          def.bPopulated ? "populated" : "not pop.",
          def.bBuffersContiguous ? "contig." : "not cont.",
          def.format.video.nFrameWidth,
          def.format.video.nFrameHeight,
          def.format.video.nStride,
          def.format.video.nSliceHeight,
          def.format.video.xFramerate,
          def.format.video.eColorFormat);
}

static int video_encode_test(char *outputfilename)
{
   OMX_VIDEO_PARAM_PORTFORMATTYPE format;
   OMX_PARAM_PORTDEFINITIONTYPE def;
   COMPONENT_T *video_encode = NULL;
   COMPONENT_T *list[5];
   OMX_BUFFERHEADERTYPE *buf;
   OMX_BUFFERHEADERTYPE *out;
   OMX_ERRORTYPE r;
   ILCLIENT_T *client;
   int status = 0;
   int filenumber = 0;
   FILE *outf;

   memset(list, 0, sizeof(list));

   if((client = ilclient_init()) == NULL)
   {
      return -3;
   }

   if(OMX_Init() != OMX_ErrorNone)
   {
      ilclient_destroy(client);
      return -4;
   }

   // create video_encode
   r = ilclient_create_component(client, &video_encode, "video_encode",
         ILCLIENT_DISABLE_ALL_PORTS |
         ILCLIENT_ENABLE_INPUT_BUFFERS |
         ILCLIENT_ENABLE_OUTPUT_BUFFERS);
   if (r != 0) {
      printf("ilclient_create_component() for video_encode failed with %x!\n", r);
      exit(1);
   }
   list[0] = video_encode;

   memset(&def, 0, sizeof(OMX_PARAM_PORTDEFINITIONTYPE));
   def.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
   def.nVersion.nVersion = OMX_VERSION;
   def.nPortIndex = 200;

   if (OMX_GetParameter(ILC_GET_HANDLE(video_encode), OMX_IndexParamPortDefinition, &def) != OMX_ErrorNone) {
      printf("%s:%d: OMX_GetParameter() for video_encode port 200 failed!\n", __FUNCTION__, __LINE__);
      exit(1);
   }

   print_def(def);

// Port 200: in 1/1 115200 16 enabled,not pop.,not cont. 320x240 320x240 @1966080 20
   def.format.video.nFrameWidth = RAW_WIDTH;
   def.format.video.nFrameHeight = RAW_HEIGHT;
   def.format.video.xFramerate = 30 << 16;
   def.format.video.nSliceHeight = def.format.video.nFrameHeight;
   def.format.video.nStride = def.format.video.nFrameWidth;
   def.format.video.eColorFormat = OMX_COLOR_Format24bitBGR888;

   print_def(def);

   r = OMX_SetParameter(ILC_GET_HANDLE(video_encode), OMX_IndexParamPortDefinition, &def);
   if (r != OMX_ErrorNone) {
      printf("%s:%d: OMX_SetParameter() for video_encode port 200 failed with %x!\n", __FUNCTION__, __LINE__, r);
      exit(1);
   }

   memset(&format, 0, sizeof(OMX_VIDEO_PARAM_PORTFORMATTYPE));
   format.nSize = sizeof(OMX_VIDEO_PARAM_PORTFORMATTYPE);
   format.nVersion.nVersion = OMX_VERSION;
   format.nPortIndex = 201;
   format.eCompressionFormat = OMX_VIDEO_CodingAVC;

   printf("OMX_SetParameter for video_encode:201...\n");
   r = OMX_SetParameter(ILC_GET_HANDLE(video_encode), OMX_IndexParamVideoPortFormat, &format);
   if (r != OMX_ErrorNone) {
      printf("%s:%d: OMX_SetParameter() for video_encode port 201 failed with %x!\n", __FUNCTION__, __LINE__, r);
      exit(1);
   }

   printf("encode to idle...\n");
   if (ilclient_change_component_state(video_encode, OMX_StateIdle) == -1) {
  printf("%s:%d: ilclient_change_component_state(video_encode, OMX_StateIdle) failed", __FUNCTION__, __LINE__);
   }

#if 1
   printf("enabling port buffers for 200...\n");
   if (ilclient_enable_port_buffers(video_encode, 200, NULL, NULL, NULL) != 0) {
      printf("enabling port buffers for 200 failed!\n");
      exit(1);
   }

   printf("enabling port buffers for 201...\n");
   if (ilclient_enable_port_buffers(video_encode, 201, NULL, NULL, NULL) != 0) {
      printf("enabling port buffers for 201 failed!\n");
      exit(1);
   }
#endif

   printf("encode to executing...\n");
   ilclient_change_component_state(video_encode, OMX_StateExecuting);

   outf = fopen(outputfilename, "w");
   if (outf == NULL) {
  printf("Failed to open '%s' for writing video\n", outputfilename);
  exit(1);
   }

   printf("looping for buffers...\n");
   do
   {
      buf = ilclient_get_input_buffer(video_encode, 200, 1);
      if (buf == NULL) {
         printf("Doh, no buffers for me!\n");
      } else {
//         printf("Got buffer: %p %lu %lu %lu\n", buf, buf->nAllocLen, buf->nFilledLen, buf->nOffset);

         /* fill it */
   read_raw_rgb(buf->pBuffer, &buf->nFilledLen, ++filenumber);

         if(OMX_EmptyThisBuffer(ILC_GET_HANDLE(video_encode), buf) != OMX_ErrorNone) {
            printf("Error emptying buffer!\n");
         } else {
//            printf("Buffer emptied!\n");
         }

         printf("Requesting output...\n");
         fflush(stdout);
         out = ilclient_get_output_buffer(video_encode, 201, 1);

         r = OMX_FillThisBuffer(ILC_GET_HANDLE(video_encode), out);
         if (r != OMX_ErrorNone) {
            printf("Error filling buffer: %x\n", r);
         }

   // Debug print the buffer flags
   if (out->nFlags & OMX_BUFFERFLAG_CODECCONFIG) {
     printf("Got buffer flag: OMX_BUFFERFLAG_CODECCONFIG\n");
   }
   if (out->nFlags & OMX_BUFFERFLAG_ENDOFFRAME) {
     printf("Got buffer flag: OMX_BUFFERFLAG_ENDOFFRAME\n");
   }
   if (out->nFlags & OMX_BUFFERFLAG_SYNCFRAME ) {
     printf("Got buffer flag: OMX_BUFFERFLAG_SYNCFRAME\n");
   }
   if (out->nFlags & 0x400) {
     // FIXME: what is 0x400??
     printf("Got buffer flag: 0x400 flag\n");
   }
   if (out->nFlags & ~(OMX_BUFFERFLAG_CODECCONFIG | OMX_BUFFERFLAG_ENDOFFRAME | OMX_BUFFERFLAG_SYNCFRAME | 0x400)) {
     printf("Got more buffer flag: %lx\n",
      out->nFlags & ~(OMX_BUFFERFLAG_CODECCONFIG | OMX_BUFFERFLAG_ENDOFFRAME | 0x400 | OMX_BUFFERFLAG_SYNCFRAME));
   }

         if (out != NULL) {
            printf("Got it: %p %lu\n", out, out->nFilledLen);
   if (out->nFlags & OMX_BUFFERFLAG_CODECCONFIG) {
     int i;
     for (i = 0; i < out->nFilledLen; i++)
    printf("%x ", out->pBuffer[i]);
     printf("\n");
   }

   r = fwrite(out->pBuffer, 1, out->nFilledLen, outf);
   if (r != out->nFilledLen) {
     printf("fwrite: Error emptying buffer: %d!\n", r);
   } else {
     printf("Buffer emptied!\n");
   }
   out->nFilledLen = 0;
         } else {
            printf("Not getting it :(\n");
         }

      }
   } while (filenumber < 184);

   fclose(outf);

   printf ("Teardown.\n");

   ilclient_state_transition(list, OMX_StateIdle);

   ilclient_cleanup_components(list);

   OMX_Deinit();

   ilclient_destroy(client);
   return status;
}

int main (int argc, char **argv)
{
   if (argc < 2) {
      printf("Usage: %s \n", argv[0]);
      exit(1);
   }
   bcm_host_init();
   return video_encode_test(argv[1]);
}

And the associated make file: 

OBJS=video.o
BIN=hello_video.bin
LDFLAGS+=-lilclient -lpthread

include ../Makefile.include

There really isn't any real application of the encoder yet, I think everyone is still waiting for the official camera module. From what I read the Pi is not able to cope with most USB camera module running at higher resolution (ie. 640x480) and will cause the Pi to crash. I personally tried a Microsoft Cinema HD and the video captured at 160x120 crashed roughly 10 seconds in. But regardless, this now allows the Pi to be used as a standalone transcoder to encode any data to h264 format. 

My VI Settings

Everyone has their own favorite editors. I use different editors at work (SlickEdit) and at home (Eclipse), but in either cases I use Vi emulation. 

The following is my Vi settings (~/.vimrc):

set nocompatible
set number
set smartindent
set tabstop=4
set shiftwidth=4
set expandtab
set autoindent
set hlsearch
colorscheme koehler
set guifont=courier_new:h10
syntax on

const char * const * ptr

Today I was tasked to Lint an imported module. Things went all for the most part, typical Linting affair: resolving unreferenced variables, uninitialized variables, potentially unbounded string copy, etc. Then I came across this gem: 

Info 818: Pointer parameter 'var' (line 123) could be declared as pointing to const

Looking at the log, I figured all I was missing was a const qualifier. Not that bad. 
Looking at the code, the offending line looked like this:

const char **values

which looks..... correct. 
I proceed onto consulting Google for the next 2 hours while scratching my head. Eventually I reviewed the C const usage: 

const char *ptr; // ptr is volatile, content of ptr is const
char * const ptr; // ptr is const, content of ptr volatile 
const char * const ptr; // ptr is const, content of ptr is also const

Looking at the code closely, **values represents a table of strings, thus: 

*table = table[0]; // pointer to the first string in the table
**table = table[0][0]; // points to the first byte of the first string in the table

The function in question performs string comparison by iterating through all strings in the table via: 

tables++; // go to the next string

But since pointer to the strings are not modified (ie. *table or table[i]), Lint was effectively complaining that the API could be further qualified as: 

const char * const * values // because *table is never modified

I then went a bit further and tried: 

const char * const * const values // let's also make table itself const

and confirmed that the following line now generates an error: 

table++; // this is now illegal

But since the function was defined as a callback to hook into another 3rd party library, I was unable to make the change as Lint suggested and had to purposely violate the const requirement to silence Lint: 

*table = *table; // Lint now thinks *table has been updated. 

But hey, I learnt something interesting today!

Android: Converting file to byte array

This came up as a requirement because I had to attach profile pictures to vCard on my eJabberd server. My first approach was to: 

1. Decode the jpg as bitmap.
2. Convert the bitmap into a byte array. 
3. Attach the byte array to the vCard, and perform the save. 

This approach, however, was inadvertently saving the .bmp version of the file instead of the jpg copy. The .bmp version of the same file was too big and causing the server to timeout on transaction. I did not realize this silly mistake until I examine the server log. 

This then raise the requirement to convert the .jpg directly to a byte array. I was unable to use some of the existing Java libraries because Android does not contain them. Googling and Stackoverflowing for a bit eventually I found the solution: 



InputStream is = new BufferedInputStream(new FileInputStream(
"/mnt/file_path_here"));


ByteArrayOutputStream bos = new ByteArrayOutputStream();


while (is.available() > 0) {
bos.write(is.read());
}


return bos.toByteArray();

Android: Running Background Tasks

In my app I'm using the asmack library as the back bone for packet sending and processing. Doing these in the context of the UI thread is not ideal as it may freeze the UI and leads to perception of app freezing. 


The better way to solve this problem is to run these tasks in the background. In general, this can be solved by having the UI thread queueing a Runnable onto an executor, and having the Runnable send a Broadcast when it finishes.


In practice, you'll need to instantiate an executor first: 

       /**
* Restart the executor pool.
* 
*/
private void initExecutor() {


m_executor = Executors.newFixedThreadPool(1);
}



Then implement your Runnable:

/**
* Logs into the server.
* 
*/
private class loginRunnable implements Runnable {


String user;
String pass;


public loginRunnable(String user, String pass) {


m_loginUser = user;
this.user = user;
this.pass = pass;
}


public void run() {


// TODO: 
                        // Implement your login routine. 


// Notify login success.
m_app.sendBroadcast(new Intent(LOGIN_SUCCESS));
}
}


To queue the login request, simply do: 
        m_executor.execute(new loginRunnable(user, pass));


After your Activity queues the Runnable onto the Executor, you should consider populating a loading Dialog indicate the request is being processed. The Dialog should be cancelled when the complete event is received in the Broadcast Receiver. 

Android: Using Preferences

Sometimes I find myself needing to cache certain little system status, such as if the user is currently logged in, what the logged in username/password are, etc. Using SQLiteHelper in this scenario is probably an overkill. This is when SharedPreferences comes in handy. In my app I implemented a simple Preferences class with the following definition: 



public final class Preferences {


/** 
* Configurable attributes. 
* 
*/
public static final String LOGIN_USERNAME = "username";
public static final String LOGIN_PASSWORD = "password";
public static final String LOGIN_STATE = "loginState";


private static SharedPreferences pref;


private Preferences() {


}


static public void init(Context context) {


pref = context
.getSharedPreferences("PREF", Context.MODE_PRIVATE);
}


static public void setBoolean(String key, boolean value) {


pref.edit().putBoolean(key, value).commit();
}


static public boolean getBoolean(String key) {


return pref.getBoolean(key, false);
}


static public void setString(String key, String value) {


pref.edit().putString(key, value).commit();
}


static public String getString(String key) {


return pref.getString(key, null);
}
}


Since the function is implemented as a static class, one can simply call 


Preferences.setBoolean(Preferences.LOGIN_USERNAME, user); 


to set the variable anywhere else in the app. 

Setting up Ejabberd

For the Android app I've been developing, I've been using OpenFire as the back end for development purposes. Openfire was chosen mostly for convenience, but I didn't plan on using it for production as it costs when number of user scales up.
I did a bit of research and settled on ejabberd for my production server. Installing the server was as painless as it can get:


sudo apt-get install ejabberd


Note that this will install the ejabberd from the repository, which may not be the latest. I did grab the latest copy, but had problem install it.


After installation completes, I had to do some custom configuration.
Stop ejabberd first:


sudo /etc/init.d/ejabberd stop


Open the cfg file:


sudo vi /etc/ejabberd/ejabberd.cfg


For debugging purposes, I enabled debug log:


{loglevel, 4}


Set the admin user:


{acl, admin, {user, "ricky", "password"}},


Set the host. Since I repurposed another PC at home, I just set the host to its IP address:


{hosts, ["192.168.1.101"]},


For my application, I also needed to enable user directory search:


{mod_vcard, [{host, "search.192.168.1.101"}]},


That's it. To restart the server:


sudo /etc/init.d/ejabberd start


To view the log, you can:


tail -f /var/log/ejabberd/ejabberd.log


Another useful feature is to dump the current database: 


sudo ejabberdctl dump /tmp/ejabberd_dump


You can Google ejabberdctl command to see the exhaustive command set. 


I'm also using asmack for the client. I'll put up some other posts in the future regarding asmack.