Sunday 29 July 2012

Opa Language Tutorial: Part 2

Continuing from part 1, I want to extend my application to actually work with some data. Let's go back to the requirements from earlier and look at how we wish to work with those and map these into the REST idea.

Firstly we require a list of regex expressions, with identifiers and descriptions [1], for example:
  • Expression visa1 describes the card number on Visa cards: ^4[0-9]{12}(?:[0-9]{3})?$
  • Expression master1 describes the card number on Mastercard cards: ^5[1-5][0-9]{14}$
  • Expression diners1 describes the card number on Diners Card cards: ^3(?:0[0-5]|[68][0-9])[0-9]{11}$
So we have a data structure of the form: identifier x regex x description, which in turn can be all represented using strings. Opa allows us to define such data structures as types:

type regexExpression = { string exprID, string regex, string description };

As we're being formal about this, let us also decide that the field exprID should be unique, ie: a primary key for referring to any given expression.

Secondly we want to store these somewhere and conveniently Opa provides us with a database mechanism. These are constructed similarly to types above but use the database keyword and database paths:

database regexDB {
   stringmap(regexExpression) /expressions

The above code defines a database called regexDB which contains a "stringmap" structure of regexExpressions and identified using the database path /expressions.

Firstly the fully qualified database path is /regexDB/expressions . Each path can be considered to be a field in the database. Some "fields" we would rather act more like tables and so we use the Opa type stringmap which is equivalent to a dictionary or hashtable in other languages. Stringmap defines the keys to be of type string and we define the value for those keys to be - in this case - regexExpressions.

Part of the REST idea was based upon the CRUD concept and indeed all well behaved databases should follow the CRUD concepts. Fortunately REST and CRUD do map quite nicely together and we can define the operations upon the database and data type we have defined:

First we have to note that the /expressions database path can either be used as is, that is there is a resource, eg: http://a.b.c.d/expressions that we can navigate to, or, because we're introduced the concept of a key through the stringmap we could refer directly to resources contained in or under, eg: http://a.b.c.d/expressions/visa1. This gives us the following table:

Verb /expressions /expressions/"k"
GET return a list of expression identifiers even if empty. Return a 200 success code return the full details of expression "k". If "k" doesn't exist then return a 404 error
POST add the requested object if the supplied key in the object doesn't exist. Return a 201 success code as well as the key "k", otherwise return a 400 error. not allowed, return a 400 error
PUT not allowed, return a 400 error modify the database with the given object if the supplied key both exists in the database and matches the key in the supplied object - return a 200 success code. In all other circumstances return a 400 error.
DELETE not allowed, return a 400 error delete the database entry with the given key if it exists in the database and return a 200 success code. In all other circumstances return a 400 error.

Let us look at the above as a check that they are a) consistent/correct and b) satisfy the various REST principles or invariants. For the GET operation the key must exist otherwise error, similarly for PUT and DELETE. For POST the key must not exist initially. In terms of REST each GET is certainly idempotent and POST, PUT and DELETE act according if accessed multiple times - this all sounds good. We should really write a much better formal specification of this, however we can leave that to the reader :-) So that's a lot of specification and now we should move on to the fun bit of programming! We now define a REST endpoint for the above and this is achieved by adding an additional case statement to our dispatcher:

function start(url) {
   match (url) {
       case {path: [] ... }: hello();
       case {path: ["expressions" | restofpath] ...} : expressionsRESTendpoint(restofpath);
       case {~path ...}: error();
   }
}

and also write some skeleton code for our new expressionsRESTendpoint() function:

function expressionsRESTendpoint(path) {
   Debug.warning("Path is {path}.");
   match(HttpRequest.get_method()){
      case{some: method}:
          Debug.warning("Method is {method}.");
          Resource.raw_status({success});
      default:
          Resource.raw_status({bad_request});
   }
}

The match statement in the start() function should be reasonably self-explanatory in that we call expressionsRESTendpoint() when we match the path of the URI such that the first entry is the "expressions". Note this is a list comprehension so that the variable "restofpath" now refers to the remainder of the list.

The skeleton code for expressionsRESTendpoint() takes a list as its parameter. The first line of this function is a debugging statement - this is Opa's way of debugging by printf: Debug.warning("xyz"). At run-time these statements are output by generated executable to the terminal (stderr?). This statement also shows a feature of Opa which is that you can enter a code block inside a string (or xhtml as we shall see later) - the use of { and } braces denotes the code block. Remember Opa is functional so that a code block will return a value, in this case it is the list in the path variable. Opa also makes the type conversion to string implicitly.

In this function we have a match statement which takes the current context of the HttpRequest to the server and extracts the HTTP method from it. This function call returns an "option"-type in Opa parlance so we need to check whether it actually does have a value or not. For Haskell programmers the equivalent is Maybe. They way we do this is to construct yet another match statement (pattern matching is your friend) and choose between whether the value exists or not. Opa uses the keyword some to find out whether this value exists or not, an in the above, assigns the value to the variable named "method".

Aside: I really need to write a proper tutorial for how match-case and pattern matching works.

Opa's match-case statements also can have a default case to catch cases where nothing is matched. A good tip is always to include a default statement, even if you think you've managed to catch 100% of cases; defensive programming!

Again we include a debugging statement to return the name of the http method. After this we generate a Resource object but this time just built so that it is just effectively an http return code - in this case HTTP1.1 200 OK or success. The default case is similar except that we return the 400 Bad Request error.

Now we can test the above code - after compiling it. To test we use curl as a way of easily inspecting the return codes and controlling what we send. If we issue the following commands:

ian@U11-VirtualBox:~/opatutorial$ curl -X GET http://127.0.0.1:8080/expressions
ian@U11-VirtualBox:~/opatutorial$ curl -X PUT http://127.0.0.1:8080/expressions/ford
ian@U11-VirtualBox:~/opatutorial$ curl -X PUT http://127.0.0.1:8080/expressions/ford/prefect


then on the terminal running our JavaScript executable (remember to compile and run!) then we'll see the output:

[Opa] Server dispatch Decoded URL to /expressions
[Opa] Debug Path is [].
[Opa] Debug Method is {get = {}}.
[Opa] Server dispatch Decoded URL to /expressions/ford
[Opa] Debug Path is [ford].
[Opa] Debug Method is {put = {}}.
[Opa] Server dispatch Decoded URL to /expressions/ford/prefect

[Opa] Debug Path is [ford,prefect].
[Opa] Debug Method is {put = {}}.         

Each group of 3 lines corresponding to the individual curl commands issued. Opa reports the dispatch anyway, the rest is from our Debug.warning commands. Note the path provided and the Path output by the debugging command as a list from the "restofpath" variable set during the match-case pattern matching.

Ok, so what we have is out REST end-point working and tested. Let's finish this off with updating the skeleton code we've written to match the http methods and write code to input the data sent to our database; for the moment we'll just implement POST and get the database running and populated.

function expressionsRESTendpoint(path) {
   Debug.warning("Path is {path}.");
   match(HttpRequest.get_method()) {     
      case{some: method}:
         match(method) {
             case{get}:
                Debug.warning("GET method called");
                Resource.raw_status({success});
             case{post}:
                expressionsPost();    
             default:
                Debug.warning("Some other method called {method}");
                Resource.raw_status({method_not_allowed});           
         }
      default:
          Resource.raw_status({bad_request});
   }        
}



and add a method to handle the post: expressionsPost(). The path isn't interesting for the post so we don't pass it. Out new function however takes the form:

function expressionsPost() {
   Debug.warning("Body is {HttpRequest.get_body()}");
    match(Json.deserialize(HttpRequest.get_body() ? "")) {
       case{some: jsonobject}:
          match(OpaSerialize.Json.unserialize_unsorted(jsonobject)) {
             case{some: regexExpression e}:
                Debug.warning("got object with fields {e.exprID}, {e.regex}, {e.description }");
//                /regexDB/expressions[e.exprID] <- e;
                Debug.warning("record loaded into database...try a get now");
                Resource.raw_status({success});
             default:
                Debug.warning("missing fields in JSON");
                Resource.raw_status({method_not_allowed});
          }
       default:
          Debug.warning("something failed");
          Resource.raw_status({bad_request});
    }
}




Now before anyone asks, yes the above does not conform to my spec (part 3 anyone?) and its full of debug stuff and a commented line referring to the database. The Debug.warning statements you can guess. The get_body() method returns the body of the call to the server similarly to get_method() used earlier.

I will now make an implementation choice: all our REST calls take their parameters as a JSON object in the body of the message. This will make the next line mroe understandable.

HttpRequest.get_body() ? ""

The ? operator is a short-cut for dealing with option types (explained earlier). If in this case a body does exist then return that, else return an empty string. The result of this is passed to the function Json.deserialize will turns whatever string is in the HttpRequest body into JSON. The following case statement:

case{some: jsonobject}



pattern matches that if we have a some JSON object then return this in the variable jsonobject, if not then this is caught by the default: clause later.

We perform a similar trick in the next match statement as shown in this code snippet:

match(OpaSerialize.Json.unserialize_unsorted(jsonobject)) {
  case{some: regexExpression e}:
    Debug.warning("got object with fields {e.exprID}, {e.regex}, {e.description }");
     ...

where

OpaSerialise.Json.unserialize_unsorted(jsonobject)

turns our valid JSON into hopefully valid Opa...if successful then we get some Opa which we coerce (type cast) into our type regexExpression and return it in variable "e". The Debug.warning statement should be self-explanatory.

Aside: yes, this could have been written neater and there might be a better way of dealing with unserialising JSON...I'd certainly like to know the latter.

If we run the above code, say with the two requests:

ian@U11-VirtualBox:~/opatutorial$ curl -X POST -d "hello" http://127.0.0.1:8080/expressions
ian@U11-VirtualBox:~/opatutorial$ curl -X POST -T "regex1" http://127.0.0.1:8080/expressions


NB: regex1 is a file containing some JSON:

{
 "exprID":"abc",
 "regex":"[abc]+",
 "description":"ABC sequence finder"
}


we see the following on the terminal:

[Opa] Server dispatch Decoded URL to /expressions
[Opa] Debug Path is [].
[Opa] Debug Body is {some = hello}
[Opa] Debug something failed
[Opa] Server dispatch Decoded URL to /expressions
[Opa] Debug Path is [].
[Opa] Debug Body is {some = {
 "exprID":"abc",
 "regex":"[abc]+",
 "description":"ABC sequence finder"
}

}
[Opa] Debug got object with fields abc, [abc]+, ABC sequence finder
[Opa] Debug record loaded into database...try a get now


Of course nothing got written into the database (line was commented out), but for the first call we can see that we hit the default match clause for an invalid JSON object. In the second we can see the body and the output of the debug statement that accesses the Opa object directly. So it seems to work.

NB: as I stated earlier, I'm not strictly to my specification here, nor am I being too strict about error handling - this is somewhat deliberate for the moment.

FINALLY, the database: We'll uncomment the line:

/regexDB/expressions[e.exprID] <- e; 

which states that in the expressions part of the regexDB which you recall was a stringmap of regexExpressions, we use the exprID field of "e" as the key and use the whole record as the value associated with this key. Simple addition of a record to a hashtable.

A quick note on deployment: Opa integrates with MongoDB. If you have MongoDB installed and running then fine, if not please refer to their documentation on this. If MongoDB is not installed then Opa downloads it - I'm not 100% clear on what it does but I like to try and keep things neatly managed so I know what is going on so I try to avoid this. I will assume therefore that you have MongoDB running on your local machine. Note the parameter to tell Opa that you wish it to use that instance of the database:

ian@U11-VirtualBox:~/opatutorial$ ./tutorial2.js --db-remote 127.0.0.1:27017
/home/ian /home/ian/.opa
http serving on http://U11-VirtualBox:8080
[Opa] regexDB DbGen/Mongo/SynchroStart Opening database
[Opa] MongoDriver.open 127.0.0.1:27017
[Opa] regexDB DbGen/Mongo/SynchroStart Db is ready
[Opa] regexDB DbGen/Mongo/SynchroStart Process 0 operations on the db wait list, start
[Opa] regexDB DbGen/Mongo/SynchroStart Process 0 operations on the db wait list, finished


So now issue the curl command as earlier

ian@U11-VirtualBox:~/opatutorial$ curl -X POST -T "regex1" http://127.0.0.1:8080/expressions

and I promise, somewhere in MongoDB lies a perfectly formed record. Which of course we can't see because we haven't written any GET statement and in all great academic traditions, this is left as an exercise for the reader.

To summarise:
  • we created a type
  • a database to store records of some type
  • extended the pattern matching to process certain paths differently
  • met option types
  • caught an http method
  • deserialised some JSON into Opa
  • started Opa with an external database (and even maybe installed and run MongoDB along the way)
  • stored a record in that database via a REST call
In the next part I want to clean up the code and make it conform to the specification, write the GET, PUT and DELETE statements, correct the sanity checking and error reporting; and try and validate as much against specification as I can.

Finally, while writing this I was reminded of this quote by Kernighan and Pauger [2]:
 “Everyone knows that debugging is twice as hard as writing a program in the first place. So if you’re as clever as you can be when you write it, how will you ever debug it?”
as I'm writing this tutorial and explaining what I'm doing I'm very quickly finding all those places where my code isn't great, my spec isn't great and more importantly forcing myself to really understanding what is going on...that's the only way you're going to get great code at the end of the day...explain it!

References

[1] Finding or Verifying Credit Card Numbers
[2] B. W. Kernighan and P. J. Plauger, The Elements of Programming Style 2nd Edition, McGraw Hill, New York, 1978. ISBN 0-07-034207-5

Saturday 28 July 2012

Opa Language Tutorial: Part 1

I've been experimenting with the Opa programming language for a couple of weeks now and, for me, it addresses various issues of writing web/cloud services really quite well. It has a bunch of features and some very nice abstractions that make the job quite well. In a nutshell it combines the features and semantics of OCaml, Erlang and Javascript and binds them together. I'm not going to draw comparisons with other languages and frameworks - I'll leave that to others, but suffice to say, I like Opa and if it does the job, in this case really quite well, then I'm happy.

So that aside I though it would be nice to present some of my results and experiences of application design and programming in Opa as a tutorial. Opa already has a good overview tutorial and there's further discussion on StackOverflow, the original Opa tutorial (a tour of Opa) and on the Opa forums, but nothing beats learning a language by actually applying it to a real problem and within the process and method structures found in practice.

So what we're going to do is develop a system or application for managing regular expressions. Just something simple enough to easily build, understand and use enough basic features to make an interesting application. Here are the requirements:
  1. Store regular expressions with descriptions
  2. Store logical sets of regular expressions with descriptions
  3. Present the data back in JSON format
  4. Have some nice reports and static front-end to this database for documentation, debugging etc...
  5. Must be REST
  6. Must be "Cloudified"
OK, the last two are architectural in nature and part of the current web/cloud zeitgeist.

The REST paradigm has some good, architectural ideas and fits in quite well with that other love of mine: formal methods. Now before you run away screaming saying that "agile" is the way to go and we don't need that stinking design stuff, agile is just a way a managing your processes such that you focus on what functionality to produce next in order to immediately satisfy the customer. In order to understand and prioritise you need to understand what you're doing and effectively communicate that - formal methods give us that clarity. This isn't a tutorial (nor argument/discussion - I've written about this before here and here) about agile and formal methods but I'll be using their principles (see [1]).

But first, let's get started with some design. REST and web design talks about resources which are access via HTTP verbs and URIs. Skipping a long thought process, let's partially tackle requirement #4 and we get something like:

Resource GET
/ Return welcome page. Always successful (HTTP 200 OK)

Let's now write some Opa.

I assume you have Opa installed and are comfortable with programming in general. Here's the entire code for this part of the tutorial:

function hello() {
   Resource.styled_page("Expressions Server - Hello", ["/resources/css.css"],
      <div>
         This server contains various regular expressions for data analysis either presented individually or by grouping and returned as JSON objects via the REST interface. See API documents for more information
       </> );
}

function error() {
   Resource.styled_page("Expressions Server - Error", ["/resources/css.css"],
      <div>
          No idea of what you want, go read the API documents for the web and REST/JSON interfaces, or better still go read the source code!
      </> );
}

function start(uri) {
   match (uri) {
       case {path: [] ... }: hello();
       case {~path ...}: error();
   }
}

Server.start(
   Server.http,
     [ {resources: @static_include_directory("resources")} , {dispatch: start} ]
);



The two functions hello() and error() should be self-explanatory above - their syntax isn't too different from Java, C++ etc. Inside however we have a single statement:

Resource.styled_page("Expressions Server - Hello", ["/resources/css.css"],
      <div>
         This server contains various regular expressions for data analysis either presented individually or by grouping and returned as JSON objects via the REST interface. See API documents for more information
       </> );


Resource.styled_page is an Opa function that returns web pages with a title, a attached files and some content.

  • The first parameter is a string containing the title of the page, in this case it is the string  "Expressions Server - Hello".
  • The second parameter is a list, denoted by square brackets and a comma separated items (we have only one item here). Each item is a string containing a link to some style resource. In this case the link is /resources/css.css which points to some (currently undefined) CSS file. 
  • The last entry is XHTML containing the content of the page.

Opa treats XHTML as part of the language and allows Opa code and XHTML to be freely mixed - cool feature! Strictly speaking XHTML in a datatype in Opa recognised by the use of opening and closing tags, in this case we're using div tags. We'll look more at what you can do with Opa and XHTML later but now we can see that this parameter in the above function simply presents us with a div section and some welcoming text.

When called Resource.style_page places all this together and returns a resource object which will be presented back to the caller as, effectively, a web page with DOCTYPE and other necessary structures. More strictly a Resource is anything that can be accessed by a URL, eg: webpage, picture, XML, JSON object etc...

Opa treats the last line of any function as the value to be returned by that function. Some languages use an expilcit return statement whereas Opa doesn't - this belies somewhat Opa's functional heritage.

Let's now jump to the Server.start() function:

Server.start(
   Server.http,
     [ {resources: @static_include_directory("resources")} , {dispatch: start} ]
);


Server.start() is Opa's equivalent of main(). In this case we're passing two parameters:
  • the first is the configuration of the server - in this case Server.http is a convenience function that returns a suitable configuration for a basic, default http protocol server.
  • the second is a list of handlers and resources for the server. This appears to be a fairly complex and sophisticated structure, here we provide two parameters: a static resource definition and a function that gets calls when the server is accessed.
The interesting thing here is the dispatch function: start(). When we call our http server, it in turn dispatches that call to a function called start() which is where the interpretation of whatever we send to the server is done:

function start(uri) {
   match (uri) {
       case {path: [] ... }: hello();
       case {~path ...}: error();
   }
}


Dispatch functions like start(), hard to believe, return objects of type Resource - we'll come to this little piece of static typing in a moment. For now start() takes a URI as parameter, this is passed by the server. So if I point my browser to "http:/127.0.0.1:8080/xyz" then the URI received by start() is "http:/127.0.0.1:8080/xyz".

The match-case takes this URI and tries to match it against the cases listed. URI's are actually structures containing fields such as path, query fields and port etc. In this case we're interested in the path. The syntax looks a little confusing but match passes the URI structure to each case statement, which then calls the path attribute of that URI. The case statement:

case {path: [] ... }:

tries to match the path of the URI against an empty list pattern. Uri paths are lists of strings if you're wondering. If a match is made then the function hello() is called.

If we pass the URI "http:/127.0.0.1/" to start() then the variable uri then the path of uri (ie: uri.path) will be empty - there is no path - and the empty list is denoted [] in Opa.

When we call the function hello() a Resouce object as described earlier is returned, this in turn is returned from the function start() back to our Server and ultimately to the program which called the server.

The second statement in the case pattern matching matches everything else. The tilde character (~) here states that is the pattern "path=path" matched which happens to be tautology and we call the function error() which behaves similarly to hello().

I guess that more or less explains the code. So let's compile it:

ian@U11-VirtualBox:~/opatutorial$ opa tutorial1.opa
Warning unused
File "tutorial1.opa", line 24, characters 14-18, (24:14-24:18 | 705-709)
  Unused variable path.
Warning inclusions.directory_empty
Directory /home/ian/opatutorial/resources is empty.
ian@U11-VirtualBox:~/opatutorial$

The command to compile is "opa" followed by the name of the file. The extension is optional but I use .opa for identification. You'll notice the compiler complaining about unused variables and empty resource directories. Take all warnings seriously:
  • in line 24, case {~path ...}: error(), we just don't use path for anything other than the pattern matching, so this is fine(ish).
  • We've included information about resources and their location but not provided anything at runtime. We don't worry about this now, I've no CSS files there anyway.
The compiler generates an executable, which in this case happens to be a file containing JavaScript and requires the existence of node.js on your system. You might need to make this file executable (I'm using Linux for these examples, never tried this on Windows nor Mac sorry) and the file contains the necessary headers to allow the various Linux shells to execute the script from the command line. Note, it may take a short while to start up

ian@U11-VirtualBox:~/opatutorial$ chmod u+rwx tutorial1.js
ian@U11-VirtualBox:~/opatutorial$ ls -l tutorial1.js
-rwx------ 1 ian ian 6171 Jul 28 10:59 tutorial1.js
ian@U11-VirtualBox:~/opatutorial$ ./tutorial1.js
http serving on http://U11-VirtualBox:8080


You can leave out the chmod and ls in future, these are there just for explanatory purposes here. Once the executable reports the http server is running and where - note the default use of port 8080, we can either conenct a browser there or use something like curl:

ian@U11-VirtualBox:~/opatutorial$ curl -i -X GET http://127.0.0.1:8080/
HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Date: Sat, 28 Jul 2012 08:03:16 GMT
Server: http
Content-Type: application/xhtml+xml; charset=utf-8
...

NB: I've cut most of the output just to show the success message (HTTP/1.1 200 OK) from the server. In a browser window it looks like so:


and if we use a longer path (any longer path in this instance will work):


As you can see both of the above correspond to the pattern matching and the calls we make to the hello() and error() functions for this statically defined content in our application.

To close the server use Ctrl+C (or kill -15, kill -9 as necessary)

So that's it for the moment...a fully functional, running web server with a couple of static pages. Not much, but it is a start...next time we'll get in to the database and some REST calls.

References

[1] Ian Oliver (2007) Experiences of Formal Methods in Conventional Software and Systems Design. BCS-FACS Meeting on Formal Methods in Industry. December 2007. London, UK

Tuesday 17 July 2012

Compiling node.js on the Raspberry Pi

I see a few people have already complied node.js on the Raspberry Pi already and so these instructions are somewhat of a repeat but here goes anyway. I've included various links below, but as you'll see we've all the same experiences more or less but it is good to see commonality.

Disclaimer: you're on your own from here...I take no responsibility for data loss, your Pi exploding or anything else.

Here's my configuration:

$uname -a
Linux raspberrypi 3.1.9+ #90 Wed Apr 18 18:23:05 BST 2012 armv6l GNU/Linux
$ cat /etc/debian_version
6.0.4
$ gcc -v
Using built-in specs.
Target: arm-linux-gnueabi
Configured with: ../src/configure -v --with-pkgversion='Debian 4.4.5-8' --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.4 --enable-shared --enable-multiarch --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4 --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --disable-sjlj-exceptions --enable-checking=release --build=arm-linux-gnueabi --host=arm-linux-gnueabi --target=arm-linux-gnueabi
Thread model: posix
gcc version 4.4.5 (Debian 4.4.5-8)


There are reports of the Pi running out of memory, so you can change the distribution between the CPU and GPU memory allocation by using a different kernel.

NB: the $ symbol is the command prompt I'm using under bash!

$sudo cp /boot/arm224_start.elf /boot/start.elf

This is explained more here: Changing RaspberryPi RAM CPU:GPU Ratio

Then I downloaded libssl-dev, after running apt-get update a couple(!!) of times.

$sudo apt-get update
$sudo apt-get update
$sudo apt-get install libssl-dev

Download and extract node.js:

$wget http://nodejs.org/dist/v0.8.2/node-v0.8.2.tar.gz
$tar xvf node-v0.8.2.tar.gz
$cd node-v0.8.2

Set the compilation flags:

$export CCFLAGS='-march=armv6'
$export CXXFLAGS='-march=armv6'
$export GYP_DEFINES='armv7=0'

Edit one of the node.js configuration files with vi

$vi deps/v8/SConstruct

Aside: use vi as emacs is now too big for any known Earth based computer - rumour has it that the cloud was only invented to gather together enough resources just to even start emacs, let alone do anything useful...(note to emacs users, this is humour!! ;-)

The lines you're looking for are found around lines 82-90 and look like this:

'all': {
    'CCFLAGS': ['$DIALECTFLAGS', '$WARNINGFLAGS'],
    'CXXFLAGS': ['-fno-rtti', '-fno-exceptions'],
},


change these by adding the -march=armv6 options so they look like this:

'all': {
   'CCFLAGS': ['$DIALECTFLAGS', '$WARNINGFLAGS', '-march=armv6'],
   'CXXFLAGS': ['-fno-rtti', '-fno-exceptions', '-march=armv6'],
},


Then somewhere around lines 170-175 you'll find:

'armeabi:softfp' : {
   'CPPDEFINES' : ['USE_EABI_HARDFLOAT=0'],
   'vfp3:on': {
      'CPPDEFINES' : ['CAN_USE_VFP_INSTRUCTIONS']
   },
   'simulator:none': {
      'CCFLAGS': ['-mfloat-abi=softfp'],
   }
},


You'll need to comment out the 'vfp3' and 'simulator' sections AND remove the trailing comma at the end of the 'CPPDEFINES' line so that it now looks like this:

'armeabi:softfp' : {
   'CPPDEFINES' : ['USE_EABI_HARDFLOAT=0']
   # 'vfp3:on': {
   #  'CPPDEFINES' : ['CAN_USE_VFP_INSTRUCTIONS']
   # },
   # 'simulator:none': {
   #  'CCFLAGS': ['-mfloat-abi=softfp'],
   # }
},

NB: remember that trailing comma!

And start the compilation:

$./configure
$make CFLAGS+=-O2 CXXFLAGS+=-O2

Note the use of the -O2 option, aparently this is to get around some compiler bug which seems to result in the following errors:

make[1]: Entering directory `/home/pi/node-v0.8.2/out'
  ACTION _home_pi_node_v0_8_2_deps_v8_tools_gyp_v8_gyp_v8_snapshot_target_run_mksnapshot /home/pi/node-v0.8.2/out/Release/obj.target/v8_snapshot/geni/snapshot.cc
pure virtual method called
terminate called without an active exception
Aborted
make[1]: *** [/home/pi/node-v0.8.2/out/Release/obj.target/v8_snapshot/geni/snapshot.cc] Error 134
make[1]: Leaving directory `/home/pi/node-v0.8.2/out'
make: *** [node] Error 2


First compilation took about 2 hours 20 minutes which then ended with the above problem. I've a 4Gb SanDisk Extreme SDHC card which might be a bit of an I/O bottleneck - some people have reported much faster compilation times. Anyway, successful compilation and linking took about 2 hours 40:

$ ls -l out/Release/node
-rwxr-xr-x 1 pi pi 9475651 Jul 17 20:10 out/Release/node

and running a simple server (found via Felix's node.js beginner's guide) :






:-)

References:
  1. Tom Gallacher's page on Node.js
  2. doowttam's posting on the R-Pi forums about nodejs 0.8.1
  3. Elsmorian's Nodejs on Raspberry Pi
  4. Cannot compile node.js 0.8.2 under CentOS 6.2 - compiler optimisation errors: use -O2 option for CFLAGS and CXXFLAGS
  5. Node.js compile fails in Amazon AMI x64 - snapshot.cc - pure functional method call - more on the compiler optimisation flags.
  6. Issue 2093: fix "pure virtual method called" error with gcc 4.4.6 + -fno-strict-aliasing (original link)

Wednesday 4 July 2012

Higgs Found

Fantastic news from Cern regarding the Higgs this morning - congratulations to all involved. Here's the press released:

CERN experiments observe particle consistent with long-sought Higgs boson

Geneva, 4 July 2012.

 At a seminar held at CERN1 today as a curtain raiser to the year’s major particle physics conference, ICHEP2012 in Melbourne, the ATLAS and CMS experiments presented their latest preliminary results in the search for the long sought Higgs particle. Both experiments observe a new particle in the mass region around 125-126 GeV.

...(see link above for full release)


Sunday 1 July 2012

R-Pi and the mainframe

As far as I can work out I'm probably the second person to do this, and the first in Finland. Andrew Back's blog has an Raspberry Pi running Hercules, and in a later post SimH and VMS.

The Hercules mainframe emulator compiles successfully under Debian on the Raspberry Pi. With a few minor tweakings to get gcc to accept the armv6l parameter to the --mcpu and --march options. Surgery to the config.status file removing every reference to armv6l worked (not a pretty solution!).

Compilation took around 90 minutes with about 10-15 for the make install at the end. Typing "hercules" at the command prompt gives:


NB: Ignore the date, I was too lazy to set the R-Pi's clock..

So hercules compiles and runs. Now just waiting for the download of Volker Brandke's Turnkey MVS 3.8j distribution via Jan Maynard's IBM public domain software collection and in an hour or so should have MVS running on an emulated IBM mainframe on an 32 euro (in P&P) single board computer. Anyone fancy doing the performance calculations between the R-Pi and the original mainframe?

Why you may ask? Well, a credit card sized 1970's mainframe is...well...ok, I have no good answer apart from it's fun...which is what the R-Pi is all about...