I’ve been wanting to do this myself, but haven’t found the time. Mark is going to do this as a part of his distributed systems cource. I’m going to help him both on a conceptual and implementation-wise. I’ll update when there’s more to know. Untill now, I’m happy to know that Sirkel may be the first DHT I know of that implements real content authentication!
I’ve been working on a parser for the OpenAir format this weekend. As usual I like to work in Haskell. The Parsec library makes sense when creating parsers. You build up a parser from simpler parser, and in the end , you’re sitting with a parser for a whole language, it’s known as the parser combinator approach.
Parsing OpenAir gave me some special challenges. OpenAir is not what I would call an data format, but more a program. It’s a list of instructions, and if you follow them, you will create an airspace. This is very non declarative, very non haskell. Therefore it makes it a challenge to parse it in haskell. Thanks to parsecs mutable state, everything got together.
I’m starting to think that OpenAir really needs a touchup though. OpenAir has choosen a data is programs approach, it feels really unatural. Maybe using YAML as a carrier format together with a schema specification on what attributes an airspace has would be more fruitful?
The instant benefit with my new parser is: Germany! We’ve now got complete and official coverage of Germany’s airspaces. That’s no small feat for a hobby project! Hungary is coming along nicely to thanks to Ákos’ work on an eAIP parser. That will allow us to parse airspaces for all countries that has an eAIP, and that’s actually some =)
I’ve done some work on migrating data from the master OSM database over to our OAM database. The tricky thing here is that when you download something from OSM, it expects any modifications etc. to be uploaded to OSM, not another database. To work around this problem I’ve written an tool called OsmXmlTool in Haskell. Put simply it does this. Parse an OSM xml file. Create an internal representation of the data in it using haskells Algebraic Datatypes. Then it modifies this representation following several rules; all entities has to get new unique id’s etc. Afther that, the datatype is converted back to XML. This data can be loaded into JOSM, fine tuned and uploaded.
To give a idea of how much work this tool saves us. Today i downloaded a bounding box of Norway with all aviation related features. I put it trough my tool, and after that into JOSM. Some minor tweaks to some tags and uploaded. Took me around 10 minutes. In those 10 minutes i had populated the database of 3000 new entities, runways, hangars, helipads, taxiways you name it. Haskell is awesome, OSM too!
If you wonder how anything is done or have any questions, pop a comment! I’m maybe thinking of making the tool generic, so that it can be used in any project that want to migrate OSM data into their own database, tell me if you’re interested.
Had a lecture in quantum mechanics today, we’ve been starting to get serious and are now working with arbitrary 1 dimentional potentials. While doing the work I feelt desperatly in need of some visualization. Therefore I packed together a little simulation in haskell this afternoon. Now, Haskell is the right tool for this kind of job. The whole program is just 63 lines of code, and they’re ugly too. I bet that it could be crammed down if one wanted. Anyways, Haskell is so great at this stuff for many reasons. For example it lets me define Numerical instances of functions, that mean that i can add two functions together, before I give them the arguments. This is great for building up complex mathematical functions. In the future I’d like to extend this example to generate a wavefunction for an arbitrary potential. This requires solving differential equations though, I don’t think Haskell’s typesystem is up for that yet. Anyway I’d be interested in ways to express maths, especially differential equations in Haskell, if you have any ideas, tell me!
The video represent the wave function of a quantum particle in a box. Unlike normal things in day to day life, quantum partilces doesn’t have a defined possition, they have a probability of being at given possitions. The y axis of this animation represents the probability of finding the particle at a given point. This specific example is the superpossition of the 4 first exited states of a box particle. As you see, the “possition” that is where you would expect to find the particle changes with time. That mean we have a moving particle. That might not sound strange, but if i say that each of the 4 exited states in themselves are stationary, they don’t move at all, then things start to become interesting. The thing is that by superpositioning differnt static states, you can create a new state, that is dynamic, and thus move; exciting!
The video is generated from a series of pictures output by my haskell program. Since I don’t know of any good way to create interactive plots in haskell, and this was an 2-hour evening project i decided to go for: Loads of images -> ffmpeg -> video -> profit!
So how does this OpenAviationMap tick? I’ll try to explain as condensed as possible.
The main part is a web application written in Ruby called Rails Port. It handles users, and the map API. This is an XML API that lets users send in changes to the map.
To edit the map we have JOSM. It’s a Java application written for OpenStreetMap. It lets you draw things on the screen and give it tags. Then you can press upload, enter username and password, and JOSM posts your beautiful drawing to the API.
The Rails Port application then receives this and stores it in a PostGIS enabled PostgreSQL database.
All changes to the map are incremental. That means that you can individually revert changes, much like on Wikipedia or VCS’s like GIT or SVN. This gives us the power to let everyone with an account edit the map by default, because no one can do irreplaceable damage.
To view the map I’m using OpenLayers, written in JavaScript. I serve the OpenAviationMap data from the database via some custom code as WFS to OpenLayers which displays it appropriately over (currently) a Google physical base map.
Ever since I took my private pilots license the thought of how old and outdated my goverments aviation data is distributed have been coming back to me once ever now and then. The thought started to grow in me and today I’ve come far enough to say that I’ve started to do something with the problem. I’ve started an open source mapping project. You’ve might have heard of Open Street Map, the idéa goes like this:
Let’s create a website and software infrastructure that makes it really easy for the lay man to create a map. Then, instead of having to bother about drawing all these ways, junktions and houses, we’ll just have to create the software that makes it really easy to do.
The thing is that they’ve come real far on this thought. Just take a look at OpenStreetMap and look at your local area. Chanses are you see a really good map of roads, important houses etc. Now comes the part that makes this map so different from all the other maps around, Google, Bing etc.
This map is make by volunteers that just enjoy the feeling of creating something important and complex.
The map is completely free, you can do whatever you want with. You always have to share what you do with it under the same license. This makes sure that if someone does a really smart thing; everyone benefits from it which I think is really neat!
What I’ve done is blatantly copy OpenStreetMaps philosophy and software architecture and used it for aviation data. To take a look at the page go to OpenAviationMap.
It lets hobby aviators around the world stick their heads together and create a world map for flying!
Already before things are stable Germany has a load of data, thanks to Udo Geschonneck who’s done a great job adding the navaids and started some aerodromes there.
If one man can create that much data in just a few days, then creating a world map maybe issn’t such a crazy idéa afterall.
I dag kom delene fra DIY-drones! Dette blir kult. Det virker som det er relativt lett loddearbeid. Kun å lodde noen headers på det hele og så er jobben gjort. Pakken fra HobbyKing er enda ikke sendt siden det mangler en del på lager, men det blir vel etterhvert.
Har også vært i kontakt med Omega Verksted som er en gjeng oppe på NTNU som har tillgang til et eget elektronikkverksted, kult! Jeg skal få lov til å sette sammen quadcopteret der og bruke deres loddeutstyr etc. I tillegg er det jo kjekt med masse smarte folk rundt en som har peiling på alt det jeg ikke kan om elektronikk =)
Jeg har startet et nytt prosjekt, bygging av et quadrocopter. Dette er en avansert fjerstyrt flyvende platform. Her er en video som forklarer litt:
Jeg gikk for Ardupilot Mega fra DIYdrones siden denne kjører på Arduinoplatformen som gjør at jeg kan programmere til den selv. Motorer og motorkontrollere kommer fra HobbyKing. Jeg planlegger å bygge rammen selv og har fått tilgang til et verksted oppe på NTNU. Her er handlelisten:
Ardupilot har 6 dof IMU med 3 akse akselerometer og 3 akse gyroskop, med dette kan man bestemme hvilken orientering quadrotoren har til enhver tid. I tillegg har den barometrisk trykksensor, denne forteller relativ høyde, 5Hz GPS som gir absolutt possisjon, og magnetometer som gir informasjon om rotasjon om vertikal akse. Med disse sensorene har Aprdupilot ganske god kontroll over hvor den er. Jeg skal selvfølgelig ikke bare bruke quadrotoren som et leketøy. Det jeg har mest lyst til å gjøre er å programmere UAV rutiner for den, slik at den blir helt selvstyrt. Målet er altså å kunne slippe kontrollen å la den kjøre showet selv. Ardupilot har allerede endel UAV funksjoner, for eksempel har den en egen backup-chip som er failsafe pilot. Denne kan du programmere til å fly tilbake til et kjent sted og lande av seg selv hvis noe skulle gå galt. Alt i alt tror jeg dette kan bli et ganske spennende robotprosjekt, vi får se hvordan det går videre!
The first release of Sirkel was released today! It contains an implementation of the Chord DHT, and a simple storage layer with fault tolerance and replication, much similar to DHash. The DHT uses the not yet released Cloud Haskell framework with it’s Erlang primitives for concurrency. It’s still in it’s very early stages but I’ve done a lot of testing and it should be reasonably stable.
Right now, this is what the DHT can do: You can run it on a local network, LAN, and in theory also on hosts directly connected to the internet, no NAT. Each node gets responsebility for a part of the key-space in the DHT. Then you can “put” and “get” all objects that are instances of Binary in and out of it. If a node leaves or disconnects, other nodes dynamically takes over the responsibility of the keys to the left node and no data is lost.
The protocol is not secure, and there is no authentication, that is, you have to trust all nodes is your system. This is an area of research and it would be really exiting to implement some sort of security into the DHT.
Now that the DHT layer is complete a lot of new doors have opened. How cool wouldn’t it be to make a distributed file-system with no single point of failure, no master controller? That’s one of the things a DHT can achieve, it’s structure is totally flat, and no-one is better, has more knowledge or power than anyone else. If you think distributed computing is fun, and would love a hobby project, join me in discovering what can be done with a DHT in Haskell!
After browsing Reddit one day I discovered a cool to me unknown algorithm, Barnes & Hut. I found a explenation of it here. I got some spare time now because of a flu/fever attack so I thought why not try to make something fun out of it. I setteled on a n-body orbital simulator with collisions. The algorithms where quite easy to jot down in Haskell, but I really struggeled on the visualization. In the end I did it the easy way. Each step, an SVG is created. Then a little imagemagik and ffmpeg magic makes a movie of it. I’m quite happy with the end result but I’d love to get better visualization, maybe OpenGL. If someone has an idea mail me =)
You can get the code from my github.
Here’s a video: orbit
Update: I’ve managed to hack together an OpenGL visualizer. It’s not very quick but it works for the purpose of demonstrating. I’ve also done some optimizations and profiling to get the code to run a bit faster, the updates source is on github.
PS. This post, and the code is written during a fever, so no refunds for eye-strain, errors etc. :P
