I’ve been keen for a while to make some custom controls for Flight Simulator X so that I can control things like lights, flaps, gear, engine, etc. After a few hours searching around for various solutions, I’ve found the Precision Joystick Controller BU0836. It’s a USB joystick controller than allows you to connect up to 10 potentiometers and 32 buttons/switches. It’s pretty much exactly what I’m looking for and looks fairly easy to work with. It comes in at about $60NZD which is cheap enough that I’ll seriously look at buying one soon (after I get a yoke and some pedals) and perhaps blog about my experiences using it.
If you’re running a web server using Apache 2.x and Ubuntu or Debian, you can save a lot of bandwidth by turning on mod_deflate. So just how does it work and how much data can you actually save?
To use trinitynet.co.nz as an example, the main page is 27.49kb with mod_deflate turned off. Once we turn it on, the page sizes reduces dramatically to 7.11kb. That’s a decrease of 74%. You can imagine what might happen if you did this on a large website!
The way it works is that when an HTTP request comes in, mod_deflate compresses the result before sending it to the client that requested it. There is some CPU overhead in doing this of course but its well worth it. Because the content gets to the user quicker, you may even notice a decrease in page load times. Generally its a good idea to only compress HTML, XML and text files; you can configure mod_deflate to compress Javascript, but this is known to cause problems with Internet Explorer. To change the mod_deflate configuration, simple edit /etc/apache2/mods-enabled/deflate.conf. On Ubuntu the default configuration is:
AddOutputFilterByType DEFLATE text/html text/plain text/xml
So how do you enable mod_deflate in Ubuntu or Debian? Easy:
Thanks to the work of some inviduals, we have managed to get postgresql-8.3-postgis included in Hardy just days before release. Thanks to all those involved. Without this package, we would have had to use PostgreSQL 8.2 instead of PostgreSQL 8.3 if we wanted PostGIS support using Ubuntu Hardy. This is a major achievement! To give you a good idea of the differences between PostgreSQL 8.2 and 8.3, you can take a look at the PostgreSQL feature matrix. You can view the Ubuntu Launchpad bug report here.
A little while ago I managed to obtain coordinate data for all the airfields and helipads in New Zealand. I imported it into a PostGIS-enabled database and since then I’ve created a script to export the GIS data as KML and output it to Google Maps. You can view it here. Something that I found while making it is that Google Maps is quite fickle about the validity of the KML file that you give it. To validate your KML file, Feed Validator is very handy.
I was modifying one of my queries for Electorate Finder this morning and came across a good example of why query optimisation and query design is important, especially on large datasets. I had a query like the following:
SELECT name, type FROM electorates WHERE
ST_Within(ST_GeomFromText('POINT(174.388544 -36.249662)', 4326), electorate_geom)Basically what this does is creates a point using ST_GeomFromText() and uses the ST_Within() function to see whether the point exists within a geom (in this case, an electorate boundary). The point is created using the WGS84 datum (SRID:4326) and because I was reverting my data back to its original format, New Zealand Map Grid (SRID:27200) I needed to use the ST_Transform function to transform my data into the same SRID. I ended up with a query like the following:
SELECT name, type FROM electorates WHERE
ST_Within(ST_GeomFromText('POINT(174.388544 -36.249662)', 4326),
ST_Transform(electorate_geom, 4326))That query achieved everything that I needed it to, but I noticed the runtime in PgAdmin: 2,032.630 ms. 2 seconds? Doesn’t that seem rather excessive? Then I realised what I had done in the query I had created. I was transforming 8MB of coordinates into a different SRID just so that I could check whether one point existed inside any of them. I’m surprised that it only took 2 seconds!
Wouldn’t it make much more sense to just transform the POINT that I created to the SRID of my electorate data? Like this:
SELECT name, type FROM electorates WHERE
ST_Within(ST_Transform(ST_GeomFromText('POINT(174.388544 -36.249662)', 4326), 27200), electorate_geom)The runtime of this query is a LOT better (153.026 ms) and the result is exactly the same.
In order to store GIS data in a PostgreSQL database, you first have to install PostGIS and then make the database spatially aware. For this example I am using Ubuntu Hardy. First, install PostGIS:
apt-get install postgis postgresql-8.3-postgis
Now create a database and a user to go with it. I’m assuming that you know how to do this already. Let’s say that your database and its user are called ‘boundary_data’.
First we need to enable PL/pgSQL and import the PostGIS functions into the database. The SQL files that we use in the next few commands should exist in ‘/usr/share/postgresql-8.3-postgis/’. Run the following on the command line as the ‘postgres’ user.
createlang plpgsql boundary_data
psql -d boundary_data -f lwpostgis.sql
Next import the ‘spatial_ref_sys.sql’ file. This contains identifiers for all the existing coordinate systems. You’ll need it if you’re going to be reprojecting your coordinates to another coordinate system.
psql -d boundary_data -f spatial_ref_sys.sql
Finally, we need to grant access to the PostGIS functions and the ‘spatial_ref_sys’ table to our user ‘boundary_data’. Run the following as the user ‘postgres’:
grant all on "geometry_columns" to boundary_data;
grant select on "spatial_ref_sys" to boundary_data;
Now your database is spatially enabled! If you have GIS data that you’d like to import and it is in the ShapeFile format, you can look at ‘shp2pgsql’.
Posted in GIS, PostgreSQL
|
Tagged GIS, PostgreSQL
|
I’ve been playing around with PostgreSQL and PostGIS lately and decided that I wanted to create something with them. I found out that the Department of Statistics provide GIS versions of the electoral boundaries and so downloaded the boundaries and started having a play. After teaching GIS to myself for the last few weeks I managed to come up with my first tool: Electorate Finder.
By supplying Electorate Finder with an NZ address, it will provide you with the General and Maori electorates that the address exists in. It does this by using the Google Maps Geocoding API to turn the address into a set of coordinates and then it searches for the coordinates inside the electorate boundaries. It’s a lot simpler than it sounds, weighing in at only ~130 lines of code (and a 20MB database).
Posted in PostgreSQL
|
Tagged GIS, PostgreSQL
|
I’ve been playing around with Django for the last couple of days for a project that I’ve had in mind for a little while. So far its been going quite well and I thought I had better import the project into Subversion. When you run the webserver that comes with Django, it compiles the project files and stores them as .pyc files inside your project directory. Obviously you don’t want these in your Subversion repository. The way to deal with this is to run this command on each folder where you want the files to be ignored:
svn propset svn:ignore "*.pyc" <directory>
Subversion will then ignore any .pyc files that are created while you’re developing your project.
Posted in Django, Tips
|
Tagged Django, Subversion
|
When you’re modifying your Exim configuration, for example setting up spam filtering, virus scanning or DNS blacklisting, it’s common practice to connect to Exim over telnet to see whether your new configuration works as expected. While this works well, you can’t actually see what Exim is doing behind the scenes and thats not particularly useful when you’re debugging something that is complicated.
However, there is a little known feature in Exim that lets you run a fake SMTP session with, just as if you were connected over telnet, but it provides a full debug log! To use it, you have to pick an IP that Exim will think you’re connecting from. If you picked the IP 127.0.0.1 for example, run the following command:
exim4 -bh 127.0.0.1
That will display the following:
**** SMTP testing session as if from host 127.0.0.1
**** but without any ident (RFC 1413) callback.
**** This is not for real!
>>> host in hosts_connection_nolog? no (option unset)
>>> host in host_lookup? yes (matched "*")
>>> looking up host name for 127.0.0.1
>>> IP address lookup yielded localhost
>>> gethostbyname2 looked up these IP addresses:
>>> name=localhost address=127.0.0.1
>>> checking addresses for localhost
>>> 127.0.0.1 OK
>>> host in host_reject_connection? no (option unset)
>>> host in sender_unqualified_hosts? no (option unset)
>>> host in recipient_unqualified_hosts? no (option unset)
>>> host in helo_verify_hosts? no (option unset)
>>> host in helo_try_verify_hosts? no (option unset)
>>> host in helo_accept_junk_hosts? no (option unset)
220 xxxx ESMTP Exim 4.63 Tue, 31 Jul 2007 14:32:17 +1200
From then on, you just communicate with Exim like you would over telnet. No mail will actually be delivered by Exim. The debug information that Exim provides will be invaluable to you.
Posted in Mail, Tips
|
Tagged Exim
|
SANS has a good article on choosing a DNS blacklist. It’s very important that you pick your blacklists carefully. I’ve seen some business Exchange servers hosted on DSL lines that use some fairly restrictive DNS blacklists with the result being that they end up losing imporant business emails.
Personally I use only Spamhaus because I know that I can trust their lists. Combined with the other spam filtering techniques that I use, I receive virtually no spam and I don’t have any legitimate mail dropped.
