Reworking www.mov.ie

My mov.ie website has been neglected for a number of years. Its main issue is that it was not kept up to date and with Official Movie websites coming and going a lot of the links that it holds are no longer valid. Even worse, when a movie producer lets the domain lapse it is quite often grabbed by a domain camper and potentially redirected to some nefarious site. This had led to the unfortunate situation where the site is now blocked by certain net filtering sites.

I have therefore decided to rework the site and at the same time bring it up to more modern technologies.

Links:

• Old Site: www.mov.ie (If your firewall/filter lets you see it)
• Prototype Work in Progress: New site: www.movie.tonyjewell.com;
  Admin site: www.movie.tonyjewell.com/Admin.html
• Google App Engine: http://developers.google.com/appengine/
• Google Web Toolkit (GWT): http://developers.google.com/web-toolkit/

Mov.ie provided service

The idea is to provide a domain for official movie web sites but with a twist - it uses sub-domains to perform searches on registered movie names. So, if you want to go to the web site for "Total Recall (2012)", you could just visit:

recall.mov.ie

As long as "recall" is unique you will be taken directly to the official movie web site. If you use a search term that's not unique then you will be given the movies that match your search. Eg:

tot.mov.ie

Will display:

• Get Him to the Greek (2010) GetHimtotheGreek.mov.ie
• How to Train Your Dragon (2010) HowtoTrainYourDragon.mov.ie
• Journey to the Centre of the Earth (2008) JourneytotheCentreoftheEarth.mov.ie
• Preaching to the Choir (2006) PreachingtotheChoir.mov.ie
• Total Recall (2012) TotalRecall.mov.ie

Of course, you can use the full movie name just stripping out all non-alphanumeric characters.

That's it - no other adverts, reviews, trailers etc as there are already excellent sites for these things - the idea is to just get people to the official site.

Mov.ie maintenance

Spotting decommissioned sites

As mentioned above, the big problem with the current site is that Official Movie Sites tend to only be maintained for a limited time. Movie producers tend to register a special catchy domain name for their movie site before release but then let it lapse after a number of years - preferring instead to just use their own sites to host the movie dvd and merchandise information.

The way of solving this is to include Host IP Address verification. On insert into the DB, the IP Address(es) for the host is also stored in the movie record. If the IP Addresses change then the movie redirect will not occur and the change will be notified to an administrator to fix. This effectively removes the movie from the system. The administrator will find the real site and fix the redirect URL.

Checking and notification will be done on access but will also be run regularly by a scheduled job - probably run off line.

Enabling Adds and Edits by other Users

At the moment, movies can only be added or modified by registered users. Also, the maintenance is not that straight forward using custom old fashioned forms. The idea is to use the users of the site to help maintain the site. The movie list will include an "Add new movie" and "Edit" on existing movies.

There will be a few simple rules (Official sites only etc) and normal users will be required to provide an email address. This will be then be sent to a moderator/administrator to verify the data and enable the change. In order to verify the email and avoid robots we will need to:

1. Use a captcha
2. Capture the details in a pending request table and generate a unique request id on the site
3. Throw the user in to an email client with the appropriate content to request the change
   (If the user has no auto-email set up then give them instructions)
4. On receipt of email by moderator, verify stored email address and request Id details with from header of email and if request is for a valid Official site redirect then enable the update.

Mov.ie technologies

The system is currently using php3 and mysql rendering html with very little dynamic aspects.

The web site will still be kept very simple but will be re-worked in to a more modern theme to work well with touch devices as well as desktop PCs. It does not need to be too fancy but will need:

• Heading
• Add button (that opens a popup dialog for add)
• Search Box
• List of Movies matching search term (Updated dynamically)
• Index Listings
• Contact/About/Copyright etc.
• Each Movie entry will be clickable to get to official site and hold:
  Title / Year / Full ".mov.ie" name
  Edit Button (that opens a popup dialog for edit)
  (and possibly "Search IMDB" button).

The goal is to get it in to the cloud and I am currently leaning towards Google App Engine as I have experience of writing Java / GWT and we all know Google's infrastructure and hosting technologies are rock solid. App engine also supports Custom Domains, Wild card sub domains, authentication, Internationalisation etc etc.

.Movie Top Level Domain

The ".movie" Global Top Level Domain has been promised for some time (since before 2000) but has never materialised. Even if this does happen this site will still be viable as it will be seen as an alternative searchable site.

Alzheimer Reminder Device

A friend of the family has an elderly Mother who is still living alone but has early onset Alzheimers. She now needs to be reminded when to take her normal medication but often forgets  the time of day and what she should be doing. Thus we need a device that cycles through the significant events of the day with pictures, prompts and time of day.

Reminder App Running on Chumby8

Hardware Choice

His suggestion was to use something like a Digital Photo frame  and have a Reminder application running on it. Such commercially available frames are reasonably priced (Around EUR80) but, unfortunately, they are low capability and are generally running some kind of RISC chip with specialised firmware rather than a generic OS. Thus, to replace the functionality you would have to flash a new Firmware. The cost saving is outweighed by the effort involved in developing to the platform.

You would not use something like a Kindle, Android or Apple Tablet for this application as you will find that the target user would get confused by the other things that the device does and also by the accidental triggering of other functions via inadvertently pressing the other buttons. My Mother was once given a Kindle but just by holding it with her arthritic fingers she would accidentally touch the various buttons on the side or she would point it at the light and the screen would rotate. They are just too generic and too complicated.

Chumby

Enter the Chumby (http://www.chumby.com/). I have owned an early Chumby Classic (http://www.chumby.com/pages/chumby_classic) for some time now and have been waiting to find an application for it. It is an open Hardware, OS, Software platform developed by Chumby Industries in San Diego, CA. It is a networked (WiFi) ARM based device, running Linux and has a small touch screen with tilt sensors and a few pressure switches on the case. It has two USB ports, Speaker, headphone socket and built in microphone. It runs Flash widgets loaded from the Chumby site. It doesn't hide its internals for those who know (You can easily gain root ssh access) yet still provides an easily configurable and secure user experience - a hard thing to get right.

Overall a great device well designed, with great infrastructure behind it. A big, Thumbs Up to the Chumby guys for their early innovation in this space that is now filled with many ARM based linux devices including Android devices, Arduino (http://www.arduino.cc/) and lately Raspberry Pi (http://www.raspberrypi.org/).

The cream on the top - and this is what makes this device ideal for this application - is that there are hooks in the boot process that allow you to configure and even replace the standard chumby console. Its as simple as creating a .swf application, loading it onto a root of a USB Stick or Flash memory card (For the later devices) using the name "controlpanel.swf". The device will then boot and only run your application.

Software

Thus, using only Open Source tool chain mtasc (.as to .swf compiler) swfmill (.swf resource linker) and gnash (GNU's Flash runner) I wrote a prototype application using simple Flash Action Script 2 that takes a schedule from an XML text file and displays reminders with pictures. It's early days but a running version can be seen online here: http://www.cregganna.com/ReminderApp/ReminderApp.html

Here is a little video of it running on my Chumby 8 in test mode.

As I say, an early prototype but it displays the basic idea. Note - it is running in test mode so the schedule is not running true time - it is just ticking every minute to start with then I trigger the changes myself by touching the screen.

The Schedule for this example is here: http://www.cregganna.com/ReminderApp/ReminderSchedule.xml

Going beyond a Prototype

The best target device would seem to be the Insignia Infocast 8 or Chumby 8 (which I think is no longer produced) as it is an 8 inch frame with a memory card slot.

Improvements and moving it forward to a fully usable application would possibly include:

1. Providing audio to play for each message - set up in the same way as the pictures.
2. Changing the idle screen - it is currently a pure white screen with the Hour:Min of the next schedule in the top right. We could have a black screen or just show pictures from a folder on the memory card.
3. Allow a cancel / confirm action when the chumby top button is pressed or the screen is tapped. This would be used to dismiss the message and go back to the idle screen. We could even have a better way of confirming that the pills have been taken etc.
4. Allow for pill sequences instead of pictures of all pills - ie take red pill, hit button, take red pill, hit button, take blue pill hit button, With Audio this might work quite well. It could repeat audio if it gets no confirmation
5. The chumby device is actually networked - there would be opportunities to remotely review the user's responses, alter the schedule and even talk to the user directly.
6. ...

To use the application one would have to:

1. Purchase a compatible device - Insignia Infocast 8 seems ideal as it is a Chumby clone. I used a Chumby 8 and Chumby One.
2. Purchase a USB Key: Any size would be OK - but if we were to make it do the Picture display in idle then the bigger the better. I went for a 4Gb Cruzer Fit as it is compact and does not extend beyond the back of the device.
3. Unzip the application (ReminderApp.zip)  on to your usb key.
4. Take pictures of the events and pills that need to be taken at each time and put them on the usb key - probably best to give them relevant names.
5. Edit the ReminderSchedule.xml for your schedule and refer to the pictures you've taken.
6. Boot and configure your device to connect to your network - you'll need a network to allow the date/time to be set correctly initially.
7. Turn the device off.
8. Stick the usb key in the device and boot.

More Pictures:

Scrolling text in Flash

Interestingly this proved hard to implement. There was lots of rubbish implementations on the internet using timers and the like so here is the implementation I put together base on MovieClip.scollRect.

function createScrollingText(container:MovieClip, fieldName:String, depth:Number,
    x:Number, y:Number, width:Number, height:Number,
    content:String) {

    container.createTextField(fieldName, depth, x, y, width, height);
    var field:TextField = container[fieldName];

    field.multiline = false;
    field.wordWrap = false;
    field.autoSize = "left"; // Needed to force render of whole string
    field.antiAliasType = "normal"; // Needed to allow scrolling whole string
    field.gridFitType = "none"; // Needed to force scrolling whole string
    field.text = content;

    var format:TextFormat = new TextFormat();
    format.font = "sans";
    format.size = height;
    format.bold = true;
    format.color = 0x000000; // Black
    reduceToHeight(field, format, height);

    // Utils.traceTextField(fieldName, field);

    if (field.textWidth <= width) {
        // trace("No Scroll required");
        // Just center it
        field.autoSize = "center";
        return;
    }

    // Double up text to make it look smoother
    field.text = " -- " + field.text + " -- " + field.text;
    // Reapply format
    field.setTextFormat(format);

    // Scroll by using horizontal scroll rectangle
    var scrollRegion:Rectangle = new Rectangle(x, y, width, height);
    var scrollLimit:Number = -1 * (field.textWidth / 2);
    container.onEnterFrame = function() {
        var newX = field._x - 1;
        if (newX < scrollLimit) {
            newX = 0;
        }
    field._x = newX;
    }
    container.scrollRect = scrollRegion;
}

Java Interfaces in Google Protobuf

Google Protobuf: Excellent idea, well executed but only if.....

[I have put this project on hold as I'm occupied with something else. I was concerned with enums in messages and composition of messages. I also came across Protostuff which has its own .proto interpreter and uses reflection to inject messages in to POJOs. You can also use Annotations to define the .proto message spec. TJ]


A proposal to add interface decoration to Protobuf generated messages.

Background

Google Protobuf is a language that defines how one encodes messages on a transport.

http://code.google.com/p/protobuf/

Like everything Google - it is well thought out, not over designed and intelligently implemented. It is target language agnostic, efficient and allows for managed upgradeability.


Except: 
The .proto message compiler (protoc) only creates rich (final) Message classes. These contain the immutable messages and access to builders for creating new messages.

Java Interfaces

Perhaps the best language feature of Java (there are many more...) is the Interface. It is a traditional implementation-free definition of the contract that an implementing object must provide.


It has a number of advantages including:

• Using an interface means that you can change the implementation but maintain the contracted functionality;
• You can write collaborators that work on objects that implement the interface without any other knowledge or dependencies on the implementation;
• When testing classes that use the interface you can stub easily stub or mock out the interfaced object;
• And many more.

Using protobuf generated message classes in the real world

The only problem is that a protobuf Message does not implement a simple interface - typically when you are receiving messages over a transport - you want to pass the messages on to handlers that know nothing about how they were encoded over the transport - indeed you might be handling the same message from different sources. OO would suggest that you're handlers would only require objects that allow access to he properties that they require to do their work. As the Protobuf Generated Message class has no means of attaching an interface one has to take one of two routes:

1. Create a new Object from the Message (probably using a factory) that implements your interface; or
2. Create a delegate that when you give it an object it extracts the relevant part.

The first doubles your number of objects created (one for the message and one for interface implementation. The second requires that extra complexity on the delegate and an extra call per property retrieved.

Proposal - decorating Generated Messages with Interfaces

My suggestion is to allow one to write an interface to encapsulate the properties you require from any message you will be processing and then decorate the Generated Message with that interface.


For instance, if we have a message in spotrate.proto:

message SpotRate {
 required uint64 timestamp = 1;
 required string currencyPair = 2;
 required double rate = 3;
 required string trader = 4;
 // ... and so on
}

Then if our message consumer was only interested in the currency pair andrate we would create an interface in ISpotRate.java:

public interface ISpotRate {
 String getCurrencyPair();
 double getRate();
}

[HERE - show example new syntax and usage]

Cisco NSLU2 (Slug) death

I run a hacked Cisco NSLU2 ("Slug") NAS as a File, DNS, DHCP and HTTP Server here at home.


It is currently running Unslung (V2.3R63-uNSLUng-6.8-beta) from http://www.nslu2-linux.org/

I've had it since about 2006 and a little while back it failed showing no lights and it would not boot.


The power supply was ok with a healthy 5.3V but nothing I could do would coax it back to life.


Fearing that I would have to give up on it - I took it apart using instructions here: ??? Where are they ???


And found the following:

You can see that the two electrolytic capacitors on the right have blown. Their caps are slightly curved outwards. A quick trip to Maplins (www.maplin.co.uk) and a bit of soldering and they now look like this:

Re-assemble, apply power and all is well.


I'm just posting here in case anyone has a similar issue. It would be a shame to throw such a great little ARM computer just because it succumbed to a faulty batch of capacitors.

Velleman 8055 USB Experiment Interface Board

I have had this Interface board that I purchased from Maplin for a while and have finally got round to writing a Unix Library and Tool for accessing it via the Linux USB HID device.


Project is on Google Code:
http://code.google.com/p/velleman-usb-experiment-board-tool/


Further details will appear on the project wiki but this picture is taken after running the following command to set the Digital Outputs to 7, Analog Output 1 to 128 and Analog Output 2  to 255. Multimeter is showing volts on Analog 1 (Roughly half of 5v). It is written in C.

[root@goflex src]# ./v8055 DO=7 AO1=128 AO2=255
Board: Vendor=4303, Product=21760, Version=0, Name=Velleman  USB K8055
TS,DI1,DI1C,DI2,DI2C,DI3,DI4,DI5,AI1,AI2,DO1,DO2,DO3,DO4,DO5,DO6,DO7,DO8,AO1,AO2
1333902325382,0,46775,0,4,0,0,0,77,156,1,1,1,0,0,0,0,0,128,255

Belkin BU30x000 UPS

I have had a Belkin BU304000 UPS under my desk for a while and I have finally pressed it in to service. It is now running my Broadband router, NAS and a Western Digital 1TB MyBook.

UPS Turns off after 5 minutes on low load

Unfortunately the UPS has a "feature" where if it is "Not Loaded" it will switch off after five minutes. This is documented in here:

The UPS switches off after a few minutes on battery
The Superior Series and Active Battery Backup series UPS will switch
off after about 5 minutes on battery when the load is very low, below
approximately 50 Watt.  The UPS is therefore unsuitable to power just
one or two low power devices like modems or routers.
This is done to prevent the UPS from completely discharging its battery
when there is no computer connected with software to tell the UPS when
to shut down.  This behavior cannot be changed; the UPS is designed to
power a computer.

My NAS is a Cisco / Linksys NSLU2 ("Slug") which has been hacked to run UnSlung: (V2.3R63-uNSLUng-6.8-beta). To my mind this *is* as computer. What's more it is monitoring the UPS. Yes it is low powered but if it is talking to the UPS why doesn't the UPS stay up?

The Answer - Not so Green

Further research revealed that the Belkin UPS seems to be a badged version of the Direct UPS Vista Pro 400, 600, 800 model. In the Download page there is a document: Remove Green Function VP800 which identifies this shutdown after 5 minutes on low load as "The Green Function". Furthermore, if you don't want this function then you need to remove Resistor R15A.

So I removed the R15A 1KOhm resister and it now stays up beyond the 5 minutes.


!!!DO NOT ATTEMPT THIS UNLESS YOU KNOW WHAT YOU ARE DOING!!!
!!!THERE ARE HIGH VOLTAGES IN THE UPS EVEN THOUGH IT IS OFF!!!
also it will invalidate your warranty

References:

Belkin UPS: http://www.belkin.com/uk/activebattery/
Direct UPS documentation: http://www.directups.com/download.html
Cisco Linksys NSLU2: http://homesupport.cisco.com/en-us/wireless/lbc/nslu2
UnSlung: http://www.nslu2-linux.org/wiki/Main/HomePage
Network UPS Tools: http://www.networkupstools.org/

UPS NUT Configuration:

Plugging the UPS in to SLUG and running lsusb shows that it presents itself as a USB to Serial device:

-bash-3.2# cat /proc/bus/usb/devices
T:  Bus=02 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#=  2 Spd=1.5 MxCh= 0
D:  Ver= 1.10 Cls=00(>ifc ) Sub=00 Prot=00 MxPS= 8 #Cfgs=  1
P:  Vendor=0665 ProdID=5161 Rev= 0.02
S:  Manufacturer=Cypress Semiconductor
S:  Product=USB to Serial
C:* #Ifs= 1 Cfg#= 1 Atr=80 MxPwr=100mA
I:  If#= 0 Alt= 0 #EPs= 1 Cls=03(HID  ) Sub=00 Prot=00 Driver=usbdevfs
E:  Ad=81(I) Atr=03(Int.) MxPS=   8 Ivl=32ms

So, I installed NUT (2.4.1-1):

-bash-3.2# ipkg install nut

And configured the ups to search for that device:

-bash-3.2# cat /opt/etc/ups.conf
[belkin]
        driver = blazer_usb
        port = auto
        vendorid = 0665
        productid = 5161
        #explore # required for later NUT versions
        #pollinterval = 10
        desc = "Belking UPS"

Now running blazer_usb shows that it can talk to the device:

/opt/bin/blazer_usb -DDDDD -u root -a belkin

So, configured upsd.conf:

-bash-3.2# cat /opt/etc/upsd.conf
LISTEN 127.0.0.1 3493

hosts.conf:

-bash-3.2# cat /opt/etc/hosts.conf
MONITOR belkin "Local Belkin UPS"

and an empty (for now) /opt/etc/upsd.users
Created a start up script /opt/etc/init.d/S90ups to start:

/opt/sbin/upsd -u root
/opt/bin/blazer_usb -u root -a belkin

Now we can inspect the UPS via:

-bash-3.2# upsc belkin
battery.voltage: 13.60
battery.voltage.nominal: 12.0
beeper.status: enabled
driver.name: blazer_usb
driver.parameter.pollinterval: 2
driver.parameter.port: auto
driver.parameter.productid: 5161
driver.parameter.vendorid: 0665
driver.version: 2.4.1
driver.version.internal: 0.03
input.current.nominal: 1.0
input.frequency: 50.1
input.frequency.nominal: 50
input.voltage: 232.3
input.voltage.fault: 232.3
input.voltage.nominal: 230
output.voltage: 232.3
ups.delay.shutdown: 30
ups.delay.start: 180
ups.load: 2
ups.productid: 5161
ups.status: OL
ups.temperature: 25.0
ups.type: offline / line interactive
ups.vendorid: 0665

Notes:

• The /opt/etc/upsd.{conf,users} files must not be world readable
• Yet to set up any monitoring to do shutdown - need to solve 5 minute shutdown first
• I have also set it up be accessible from the web server on the NAS by linking "ups" in the html root to "/opt/share/nut/html" and copying "/opt/etc/upsstats-single.html" and "/opt/etc/upsstats.html" from the samples. Also linking /opt/cgi-bin/ups* to the cgi-bin:

-bash-3.2# ls -l /home/httpd/html/cgi-bin/nut
total 0
lrwxrwxrwx 1 root root 25 Mar 13 14:27 upsimage.cgi -> /opt/cgi-bin/upsimage.cgi
lrwxrwxrwx 1 root root 23 Mar 13 14:27 upsset.cgi -> /opt/cgi-bin/upsset.cgi
lrwxrwxrwx 1 root root 25 Mar 13 14:27 upsstats.cgi -> /opt/cgi-bin/upsstats.cgi

Being a responsible Public Domain software user

Why am I blogging?


Being a contract Software Engineer I daily rely on a vast array of Public Domain components. This allows the teams I work in to focus more on the business requirements rather than the low-level stuff and thus we can quickly deliver complex highly functional systems.


The problem is, though, that inevitably we come across something that a library doesn't implement or a bug in the latest version that has not yet been fixed.


I have yet to come across a client that is willing to pay for your time to implement/fix the issue and usually we have to find some other solution or implement some hacky work around.


As such, and as I am "resting" between contracts, I have decided to try and put something back in to the Public Domain community by using my own free time to address some of the issues that arose in my last contract.


I'm hoping to use this blog to document and manage this work.

Introduction

Who am I?


My name is Tony Jewell and I have been coding since 1978 when I joined the London Home Office and Metropolitan Police as a Junior COBOL programmer. I had actually dabbled a bit before then at school using a very modern Teletype machine connected via modem to a mainframe computer in a London Polytechnic.


With a brief interlude at University College London to do a degree in Computer Science and Electronic Engineering I have spent the bulk of my time as a contractor in the Financial markets in London.


My favourite languages are C and Java and for OS its Unix.


My Web sites are:
www.cregganna.com
mov.ie


My Linked in profile is: http://uk.linkedin.com/in/tonyjewell