Friday, 15 June 2018

SDXC UHS-I card with exfat FS corrupted on Android Orio (8.0)

In case anyone else hits this issue I thought I'd document it here:

Whenever I get a new phone, I buy a new mico-SD Card and transfer all my Pictures, Music and useful stuff from my old card to my new. I have just been hit by rather annoying issue. When I plugged the SDXC card in to my Moto G6 (Orio 8.0.0) it gave corrupt card with the only option be to reformat it.

Turns out that the issue is a 16 bit UTF filename in the file system - something created on my old Galaxy phone with place names in picture filenames.

The exfat (1.2.5-2) driver that I installed on my Linux laptop:
apt install exfat-fuse exfat-utils
would give the following on an fsck:
Jun 15 11:25:30 tonydell mount.exfat: illegal UTF-16 sequence
Jun 15 11:25:30 tonydell mount.exfat: failed to convert name to UTF-8
My guess is, this is the same reason the Android device reports the issue as I guess it uses the same FS driver.

So, the fix is to first find the directory that has the file:

  1. First mount your card on the unix box
  2. Run "find mountdir"
    This will fail with some error message - can't remember exactly but something along the lines of "software abort" or some such. But the key is that it will give you the directory name.
  3. The FS is now disabled on linux so:
    umount the card on linux
  4. Put the card in to a window laptop and navigate to that directory
  5. Windows seems to at least cope with the strange UTF-16 filenames and you can rename all files in the directory that have non-ascii characters in them.
  6. Put the sdcard back in your Linux box and run fsck again.
    It should complete cleanly unless you have other directories with the same issue in which case just repeat the process.
If you don't have a windows box (and why should you) you will need to be more careful on transferring the files across - first checking for UTF-16 filenames.

I think this is a potential issue with Linux exfat FS implementation - rather than failing the FS on UTF-16 convert - they should just report the error and map to "?" or some such. Its bizarre that the directory write completes but the read fails.


Friday, 26 December 2014

XVnc does not present GDM Greeter (Login) after openvpn installed - XDMCP

I thought I'd write this up so that it can help anyone else who hits the same issue.

I have a Xvnc set up on my Debian Wheezy (7) with xinetd serving on ports 5900 and up.

All was working fine until I installed openvpn and set up a tunnel to my other site.

After that, I could connect using vnc but Xvnc would only show the default X background - Xvnc was not connecting to GDM using XDMCP to present the login and complete the Display Manager startup.

After a bit of strace'ing of the gdm3 process - the server for XDMCP UDP port 177 - it became obvious that gdm was having trouble resolving the IP Addresses. It all started working when I added the tunnel's local address in the /etc/hosts.

Here's the important parts of the etc/gdm3/daemon.conf:


Here's the /etc/xinetd
service Xvnc
        type = UNLISTED
        disable = no
        socket_type = stream
        protocol = tcp
        wait = no
        user = nobody
        server = /usr/bin/Xvnc
        server_args = -inetd -desktop Acer -query localhost -depth 24 -cc 3 -once -SecurityTypes=none -x XFIXES -geometry 1280x1024 -geometry 1440x900 -geometry 1024x768 -geometry 2560x1440
        port = 5901

And as my host has the following Interfaces:

Here is my /etc/hosts:       localhost      acer      acer      acer

Tuesday, 15 October 2013

Copying music to In Car Entertainment (ICE) system with limited Memory Card capacity

My car has a decent In Car Entertainment (ICE) system but unfortunately the only way of getting music on to the Hard Disc is via a Memory Card with maximum 2Gb capacity.

I therefore have to transfer the music files piece meal. I have therefore written a bit of perl that supports the copying of files but maintains a Tag file of what files have been transferred - ie its a simple mirror program that:

  • Keeps a record of files that already been copied;
  • Restricts the files transferred to those with a given suffix (currently .mp3);
  • Fails carefully when out of space on target directory;
  • Will recopy files that have changed since original copy.
It is designed for my NAS which is an ARM based Linux box with perl installed. Anybody who wishes to use it would need to ensure they have perl installed and change the "#!" header to their perl location.

Usage is (mount/umount will probably have be done as root):

  1. Mount memory card on your linux computer
    Eg: if you're card is /dev/sdb and you have created a directory /tmp/ext, as root:
    mount -t auto /dev/sdb /tmp/ext
  2. Make sure memory card is empty:
    rm -rf /tmp/ext/*
  3. Run perl copy script calling out:
    - The tags file you want to use
    - The Source directory
    - The Target directory on your memory card:
    ./ ~/carmusic.tags /home/music /tmp/ext/music
  4. This will either complete fully or will fail with "No space left on device"
  5. Unmount the card on your linux computer:
    umount /tmp/ext
  6. Put memory card in your ICE and copy to Hard Disk selecting Replace files
  7. If step 4 failed with "No space left on device" then repeat from step 1.
The tags file looks like this:
/home/music/la_roux/la_roux/armour_love.mp3^1251554384^C /home/music/la_roux/la_roux/reflections_are_protection.mp3^1251554386^C /home/music/la_roux/la_roux/growing_pains.mp3^1251554686^C /home/music/Green Day/International Superhits/Basket Case.mp3^1235913966^C /home/music/Green Day/International Superhits/Brainstew.mp3^1235914182^C /home/music/Green Day/International Superhits/Geek Stink Breath.mp3^1235914132^C /home/music/Green Day/International Superhits/Good Riddance Time of Your Life.mp3^1235914370^C /home/music/Green Day/International Superhits/Hitchin A Ride.mp3^1235914324^C /home/music/Green Day/International Superhits/JAR Jason Andrew Relva.mp3^1235914098^C /home/music/Green Day/International Superhits/Jaded.mp3^1235914206^.
Note that the file format is:

Where Status is "." if copy is in progress and "C" is completed copy. Source:


#       This perl script is used to mirro files from one dir to another
#       It keeps a tag file of files that have been copied and the modification time
#       of the source file when copy was done so that copies
#       are not repeated.
#       It is designed to fill up a device with files to be copied until its out of space
#       You can then do what you want with the files on the device (eg copy them to another device)
#       clear the device nd run the copy again. The tags file ensures that only as yet uncopied
#       files are placed on the device.
#       I wrote this so that I can transfer my music files to my car's enterntainment system using
#       a 2Gb memory card - the maximum card size supported by the ICE.
#       Usage:
# tagFileName /home/music /mnt/extcard/music

#       Author: Tony.Jewell@Cregganna.Com
#       See:
#       Disclaimer: I take not responsibility for any damage or other nepharious effects this
#               script will have on your PC - use at yoru own risk.

use strict;
use warnings;
use diagnostics;

use File::stat;
use File::Basename;
use File::Path;

if ( $#ARGV < 2 ) {
        print "Usage: cpto TagFile Source Destination\n";
        exit 1;

my @suffixes = ( ".mp3" );
my $buffSize = 8 * 2**10;
my $tagsFile = "${ARGV[0]}";
# tags is keyed by file name and value is date/time of cp from file
my %tags;

sub loadTags {
        # Check that we can write to it
        open (TAGS, ">>${tagsFile}") || die "Unable to open ${tagsFile} for writing: $!";

        if ( -f "${tagsFile}" ) {
                open( my $TAGS, "<${tagsFile}") || die "Unable to open ${tagsFile}: $!";
                while (my $tagLine = <$TAGS>) {
                        chomp $tagLine;
                        my @tag = split('\^', $tagLine);
                        if ($#tag >= 2 && $tag[2] =~ "C") {
                                $tags{$tag[0]} = $tag[1];

# We write a start Copy to make sure there is room to write to the tags file
# This writes a pending line with status "."
sub tagStartCopy {
        my $name = shift;
        my $mtime = shift;
        open (TAGS, "+<${tagsFile}") || die "Unable to open ${tagsFile} for writing: $!";

        # Seek to end
        sysseek(TAGS, 0, 2 ) || die "Unable to seek file $tagsFile: $!";

        my $tagPos = tell(TAGS);
        syswrite(TAGS, "$name^$mtime^.\n");     # Mark as pending

        close(TAGS) || die "Unable to close $tagsFile after writing: $!";

        return $tagPos;

# Rewrite the tag line with the completed copy status of "C".
sub tagEndCopy {
        my $tagPos = shift;
        my $name = shift;
        my $mtime = shift;

        open (TAGS, "+<${tagsFile}") || die "Unable to open ${tagsFile} for writing: $!";

        sysseek(TAGS, $tagPos, 0 ) || die "Unable to seek file $tagsFile: $!";
        syswrite(TAGS, "$name^$mtime^C\n");

        close(TAGS) || die "Unable to close $tagsFile after writing: $!";

        $tags{$name} = $mtime;

sub chkDirExists {
        my $dir = dirname(shift);

        if (! -d $dir) {
                mkpath($dir) || die "Unable to create Directory: $dir: $!";

sub cpFile {
        my $fromName = shift;
        my $toName = shift;

        my($filename, $directories, $suffix) = fileparse($fromName, @suffixes);
        if ( ! $suffix ) {

        my $from = stat($fromName) || die "Unable to stat $fromName: $!";
        my $tag = $tags{$fromName};

        if ( $tag && $tag >= $from->mtime ) {
                # Already done this one

        print "$fromName => $toName....";

        my $tagPos = &tagStartCopy($fromName, $from->mtime);

        open( IN, "< $fromName") || die "Unable to open for Reading: $fromName: $!";
        open( OUT, "> $toName") || die "Unable to open for Writing: $toName: $!";
        my $buff;
        while (read(IN, $buff, $buffSize)) {
                print OUT $buff;
        close( IN ) || die "Unable to close for Reading: $fromName: $!";
        close( OUT ) || die "Unable to close for Writing: $toName: $!";

        &tagEndCopy($tagPos, $fromName, $from->mtime);

        print "\n";

sub cpDir {
        my $fromName = shift;
        my $toName = shift;

        opendir(my $DIR, $fromName) || die "Unable to read directory: $fromName: $!";

        while(my $file = readdir $DIR) {
                if ( ! ($file =~ /^[.]+$/ ) ) {
                        &cp($fromName . '/' . $file, $toName . '/' . $file);

        closedir $DIR;

sub cp {
        my $fromName = shift;
        my $toName = shift;

        if (-d $fromName) {
                &cpDir($fromName, $toName);
        } elsif (-f $fromName) {
                &cpFile($fromName, $toName);

&cp($ARGV[1], $ARGV[2]);

Tuesday, 25 September 2012


My website has been neglected for a number of years. Its main issue is that it was not kept up to date and with Official Movie websites coming and going a lot of the links that it holds are no longer valid. Even worse, when a movie producer lets the domain lapse it is quite often grabbed by a domain camper and potentially redirected to some nefarious site. This had led to the unfortunate situation where the site is now blocked by certain net filtering sites.

I have therefore decided to rework the site and at the same time bring it up to more modern technologies.

Links: provided service

The idea is to provide a domain for official movie web sites but with a twist - it uses sub-domains to perform searches on registered movie names. So, if you want to go to the web site for "Total Recall (2012)", you could just visit:
As long as "recall" is unique you will be taken directly to the official movie web site. If you use a search term that's not unique then you will be given the movies that match your search. Eg:
Will display:

  • Get Him to the Greek (2010)
  • How to Train Your Dragon (2010)
  • Journey to the Centre of the Earth (2008)
  • Preaching to the Choir (2006)
  • Total Recall (2012)

Of course, you can use the full movie name just stripping out all non-alphanumeric characters.

That's it - no other adverts, reviews, trailers etc as there are already excellent sites for these things - the idea is to just get people to the official site. maintenance

Spotting decommissioned sites

As mentioned above, the big problem with the current site is that Official Movie Sites tend to only be maintained for a limited time. Movie producers tend to register a special catchy domain name for their movie site before release but then let it lapse after a number of years - preferring instead to just use their own sites to host the movie dvd and merchandise information.

The way of solving this is to include Host IP Address verification. On insert into the DB, the IP Address(es) for the host is also stored in the movie record. If the IP Addresses change then the movie redirect will not occur and the change will be notified to an administrator to fix. This effectively removes the movie from the system. The administrator will find the real site and fix the redirect URL.

Checking and notification will be done on access but will also be run regularly by a scheduled job - probably run off line.

Enabling Adds and Edits by other Users

At the moment, movies can only be added or modified by registered users. Also, the maintenance is not that straight forward using custom old fashioned forms. The idea is to use the users of the site to help maintain the site. The movie list will include an "Add new movie" and "Edit" on existing movies.

There will be a few simple rules (Official sites only etc) and normal users will be required to provide an email address. This will be then be sent to a moderator/administrator to verify the data and enable the change. In order to verify the email and avoid robots we will need to:
  1. Use a captcha
  2. Capture the details in a pending request table and generate a unique request id on the site
  3. Throw the user in to an email client with the appropriate content to request the change
    (If the user has no auto-email set up then give them instructions)
  4. On receipt of email by moderator, verify stored email address and request Id details with from header of email and if request is for a valid Official site redirect then enable the update. technologies

The system is currently using php3 and mysql rendering html with very little dynamic aspects.

The web site will still be kept very simple but will be re-worked in to a more modern theme to work well with touch devices as well as desktop PCs. It does not need to be too fancy but will need:
  • Heading
  • Add button (that opens a popup dialog for add)
  • Search Box
  • List of Movies matching search term (Updated dynamically)
  • Index Listings
  • Contact/About/Copyright etc.
  • Each Movie entry will be clickable to get to official site and hold:
    Title / Year / Full "" name
    Edit Button (that opens a popup dialog for edit)
    (and possibly "Search IMDB" button).

The goal is to get it in to the cloud and I am currently leaning towards Google App Engine as I have experience of writing Java / GWT and we all know Google's infrastructure and hosting technologies are rock solid. App engine also supports Custom Domains, Wild card sub domains, authentication, Internationalisation etc etc.

.Movie Top Level Domain

The ".movie" Global Top Level Domain has been promised for some time (since before 2000) but has never materialised. Even if this does happen this site will still be viable as it will be seen as an alternative searchable site.

Tuesday, 11 September 2012

Alzheimer Reminder Device

A friend of the family has an elderly Mother who is still living alone but has early onset Alzheimers. She now needs to be reminded when to take her normal medication but often forgets  the time of day and what she should be doing. Thus we need a device that cycles through the significant events of the day with pictures, prompts and time of day.

Reminder App Running on Chumby8

Hardware Choice

His suggestion was to use something like a Digital Photo frame  and have a Reminder application running on it. Such commercially available frames are reasonably priced (Around EUR80) but, unfortunately, they are low capability and are generally running some kind of RISC chip with specialised firmware rather than a generic OS. Thus, to replace the functionality you would have to flash a new Firmware. The cost saving is outweighed by the effort involved in developing to the platform.

You would not use something like a Kindle, Android or Apple Tablet for this application as you will find that the target user would get confused by the other things that the device does and also by the accidental triggering of other functions via inadvertently pressing the other buttons. My Mother was once given a Kindle but just by holding it with her arthritic fingers she would accidentally touch the various buttons on the side or she would point it at the light and the screen would rotate. They are just too generic and too complicated.


Enter the Chumby ( I have owned an early Chumby Classic ( for some time now and have been waiting to find an application for it. It is an open Hardware, OS, Software platform developed by Chumby Industries in San Diego, CA. It is a networked (WiFi) ARM based device, running Linux and has a small touch screen with tilt sensors and a few pressure switches on the case. It has two USB ports, Speaker, headphone socket and built in microphone. It runs Flash widgets loaded from the Chumby site. It doesn't hide its internals for those who know (You can easily gain root ssh access) yet still provides an easily configurable and secure user experience - a hard thing to get right.

Overall a great device well designed, with great infrastructure behind it. A big, Thumbs Up to the Chumby guys for their early innovation in this space that is now filled with many ARM based linux devices including Android devices, Arduino ( and lately Raspberry Pi (

The cream on the top - and this is what makes this device ideal for this application - is that there are hooks in the boot process that allow you to configure and even replace the standard chumby console. Its as simple as creating a .swf application, loading it onto a root of a USB Stick or Flash memory card (For the later devices) using the name "controlpanel.swf". The device will then boot and only run your application.


Thus, using only Open Source tool chain mtasc (.as to .swf compiler) swfmill (.swf resource linker) and gnash (GNU's Flash runner) I wrote a prototype application using simple Flash Action Script 2 that takes a schedule from an XML text file and displays reminders with pictures. It's early days but a running version can be seen online here:

Here is a little video of it running on my Chumby 8 in test mode.

As I say, an early prototype but it displays the basic idea. Note - it is running in test mode so the schedule is not running true time - it is just ticking every minute to start with then I trigger the changes myself by touching the screen.

The Schedule for this example is here:

Going beyond a Prototype

The best target device would seem to be the Insignia Infocast 8 or Chumby 8 (which I think is no longer produced) as it is an 8 inch frame with a memory card slot.

Improvements and moving it forward to a fully usable application would possibly include:
  1. Providing audio to play for each message - set up in the same way as the pictures.
  2. Changing the idle screen - it is currently a pure white screen with the Hour:Min of the next schedule in the top right. We could have a black screen or just show pictures from a folder on the memory card.
  3. Allow a cancel / confirm action when the chumby top button is pressed or the screen is tapped. This would be used to dismiss the message and go back to the idle screen. We could even have a better way of confirming that the pills have been taken etc.
  4. Allow for pill sequences instead of pictures of all pills - ie take red pill, hit button, take red pill, hit button, take blue pill hit button, With Audio this might work quite well. It could repeat audio if it gets no confirmation
  5. The chumby device is actually networked - there would be opportunities to remotely review the user's responses, alter the schedule and even talk to the user directly.
  6. ...
To use the application one would have to:
  1. Purchase a compatible device - Insignia Infocast 8 seems ideal as it is a Chumby clone. I used a Chumby 8 and Chumby One.
  2. Purchase a USB Key: Any size would be OK - but if we were to make it do the Picture display in idle then the bigger the better. I went for a 4Gb Cruzer Fit as it is compact and does not extend beyond the back of the device.
  3. Unzip the application (  on to your usb key.
  4. Take pictures of the events and pills that need to be taken at each time and put them on the usb key - probably best to give them relevant names.
  5. Edit the ReminderSchedule.xml for your schedule and refer to the pictures you've taken.
  6. Boot and configure your device to connect to your network - you'll need a network to allow the date/time to be set correctly initially.
  7. Turn the device off.
  8. Stick the usb key in the device and boot.

More Pictures:

Scrolling text in Flash

Interestingly this proved hard to implement. There was lots of rubbish implementations on the internet using timers and the like so here is the implementation I put together base on MovieClip.scollRect.
function createScrollingText(container:MovieClip, fieldName:String, depth:Number,
    x:Number, y:Number, width:Number, height:Number,
    content:String) {

    container.createTextField(fieldName, depth, x, y, width, height);
    var field:TextField = container[fieldName];

    field.multiline = false;
    field.wordWrap = false;
    field.autoSize = "left"; // Needed to force render of whole string
    field.antiAliasType = "normal"; // Needed to allow scrolling whole string
    field.gridFitType = "none"; // Needed to force scrolling whole string
    field.text = content;

    var format:TextFormat = new TextFormat();
    format.font = "sans";
    format.size = height;
    format.bold = true;
    format.color = 0x000000; // Black
    reduceToHeight(field, format, height);

    // Utils.traceTextField(fieldName, field);

    if (field.textWidth <= width) {
        // trace("No Scroll required");
        // Just center it
        field.autoSize = "center";

    // Double up text to make it look smoother
    field.text = " -- " + field.text + " -- " + field.text;
    // Reapply format

    // Scroll by using horizontal scroll rectangle
    var scrollRegion:Rectangle = new Rectangle(x, y, width, height);
    var scrollLimit:Number = -1 * (field.textWidth / 2);
    container.onEnterFrame = function() {
        var newX = field._x - 1;
        if (newX < scrollLimit) {
            newX = 0;
    field._x = newX;
    container.scrollRect = scrollRegion;

Monday, 23 April 2012

Java Interfaces in Google Protobuf

Google Protobuf: Excellent idea, well executed but only if.....

[I have put this project on hold as I'm occupied with something else. I was concerned with enums in messages and composition of messages. I also came across Protostuff which has its own .proto interpreter and uses reflection to inject messages in to POJOs. You can also use Annotations to define the .proto message spec. TJ]

A proposal to add interface decoration to Protobuf generated messages.


Google Protobuf is a language that defines how one encodes messages on a transport.

Like everything Google - it is well thought out, not over designed and intelligently implemented. It is target language agnostic, efficient and allows for managed upgradeability.

The .proto message compiler (protoc) only creates rich (final) Message classes. These contain the immutable messages and access to builders for creating new messages.

Java Interfaces

Perhaps the best language feature of Java (there are many more...) is the Interface. It is a traditional implementation-free definition of the contract that an implementing object must provide.

It has a number of advantages including:

  • Using an interface means that you can change the implementation but maintain the contracted functionality;
  • You can write collaborators that work on objects that implement the interface without any other knowledge or dependencies on the implementation;
  • When testing classes that use the interface you can stub easily stub or mock out the interfaced object;
  • And many more.

Using protobuf generated message classes in the real world

The only problem is that a protobuf Message does not implement a simple interface - typically when you are receiving messages over a transport - you want to pass the messages on to handlers that know nothing about how they were encoded over the transport - indeed you might be handling the same message from different sources. OO would suggest that you're handlers would only require objects that allow access to he properties that they require to do their work. As the Protobuf Generated Message class has no means of attaching an interface one has to take one of two routes:

  1. Create a new Object from the Message (probably using a factory) that implements your interface; or
  2. Create a delegate that when you give it an object it extracts the relevant part.
The first doubles your number of objects created (one for the message and one for interface implementation. The second requires that extra complexity on the delegate and an extra call per property retrieved.

Proposal - decorating Generated Messages with Interfaces

My suggestion is to allow one to write an interface to encapsulate the properties you require from any message you will be processing and then decorate the Generated Message with that interface.

For instance, if we have a message in spotrate.proto:
message SpotRate {
 required uint64 timestamp = 1;
 required string currencyPair = 2;
 required double rate = 3;
 required string trader = 4;
 // ... and so on
Then if our message consumer was only interested in the currency pair andrate we would create an interface in
public interface ISpotRate {
 String getCurrencyPair();
 double getRate();

[HERE - show example new syntax and usage]

Cisco NSLU2 (Slug) death

I run a hacked Cisco NSLU2 ("Slug") NAS as a File, DNS, DHCP and HTTP Server here at home.

It is currently running Unslung (V2.3R63-uNSLUng-6.8-beta) from

I've had it since about 2006 and a little while back it failed showing no lights and it would not boot.

The power supply was ok with a healthy 5.3V but nothing I could do would coax it back to life.

Fearing that I would have to give up on it - I took it apart using instructions here: ??? Where are they ???

And found the following:

You can see that the two electrolytic capacitors on the right have blown. Their caps are slightly curved outwards. A quick trip to Maplins ( and a bit of soldering and they now look like this:

Re-assemble, apply power and all is well.

I'm just posting here in case anyone has a similar issue. It would be a shame to throw such a great little ARM computer just because it succumbed to a faulty batch of capacitors.