Drupal postcode proximity search

alan — Tue, 10 Aug 2010 16:50:50 +0000

my blog_Drupal postcode proximity search

10.08.2010

The brief: a website built around a UK postcode search. The user inputs their postcode, chooses a category to narrow the search if they want, and the results can be displayed as a list of teasers or a GMap. The teaser list needs to be sorted either alphabetically or by distance from the entered postcode. Sounds simple, right?

WRONG

I have always had the impression that the Views module wavers constantly between being incredibly useful and collapsing under the weight of its own complexity. I have used Views to create websites very rapidly, but likewise I have often gotten bogged down in its limitations, bugs and unexpectedly mysterious workings. Often it can be a case of getting 95% of the way there in an hour, then spending 2 days achieving the final 5%, and this was one of those times.

With Drupal 6, Views, Locations and GMap all being quite mature at this point I was taken aback by the problems I encountered accomplishing what I considered to be one of the more basic possible uses of a location search. Searching on Google informed me that I was not alone.

Initially I was going to write about the problems of doing a proximity search from a naked postcode, but the extremely active Drupal community have resolved this issue in the last couple of weeks. So instead I thought I would just write down a couple of the essential elements of the "recipe" that I have used to accomplish my postcode search site, in the hope that someone else out there might find it useful. Who knows, the next release of location or views might make all these comments completely obsolete!

Most of the required functionality came right out of the box with Location Views: once I had a content type associated with a location, and built a few sample nodes, it was relatively easy to create a view listing those nodes. Using "node view" allowed me to style them using Content Templates. I added a location: distance/proximity argument to the view, and this took as an argument the user-entered postcode...NOT! I couldn't work out for a long time why this argument failed to work, until I realized I was missing some small grey "helper text" underneath the settings:

So, OK, I could grumble (and I did) about how this vital info was easy to miss; on the other hand, where were they supposed to put it? Oh yes, that's right - in the comprehensive, helpful documentation for Views...oh. Anyway, let's put this down to developer blindness and move on, now that we know that the postcode argument needs to be country_postcode_distance or postcode_distance (where "distance" is the number of miles away from the postcode to return location results for).

My other argument was a category taken from a hierarchical vocabulary. I tried messing around with exposing a hierarchical-select filter in views, and very quickly ran into some horrible nightmares. To sum up, if I had done it that way there is no way I would have been able to style my form as specified by the graphic designer. So I learned how to add a hierarchical select field to my own form and passed the value of this field into the view as a simple TermID argument.

Now, the user needed to be able to switch easily between Displays in the same view (i.e. viewing the same results as a GMap or a node list) and also switch between alphabetical and distance ordering. Again, after some tinkering with trying to expose filters I realized that my form was going to get very ugly very quickly, so instead what I did was create my own links by passing arguments in the URL. So to display the A-Z list, I added a display to the View whose only difference was that there was an alphabetical sort in the sort criteria, then created a link (nicely styled as specified!) in the form http://mysite.com/myform?postcode=xxxxxx&tid=1&display=page_2. Then using $_GET I was able to choose which display of the view to display, and used views_embed_view($postcode, $tid, $display).

This also enabled the user's choices to be retained while switching between different displays of the view, since I took the $_GET values and also used them to set the values of the form fields. And in one key respect it saved my sanity, namely: sorting by distance.

This should have been easy. I had a view taking a postcode as its origin and calculating distance between the origin and each node listed. However, sorting by the distance proved a severe headache. Options were present to sort by distance from user location, distance from a static postcode, distance from a location-associated nid argument....the full list is here:

What wasn't there was an option to sort by distance from my postcode argument. This one cost me about 2 hours. I quickly settled on the need to use the "Use PHP code to determine latitude/longitude) option, but using standard views syntax to get the views arguments directly ($args[0], $args[1], etc) didn't work. Eventually I figured out that since in my particular case I was passing arguments in the URL, I could use $_GET here like anywhere else to get my postcode. Then I used location_get_postalcode_data() from the Location API reference to get my lat/lon pair. Lastly I just had to figure out that although that array came back in the form array('lat'=>xx.xx, 'lon'=>xx.xx), the return value for the sorting had to be in the form array('latitude'=>xx.xx, 'longitude'=>xx.xx). Live preview didn't work for this one (because of the use of $_GET) but when I saved the view and tried my form again, it worked! Here's the code:

Incidentally, using the Location API was invaluable when I needed to access the distance/proximity value in my Content Template. Again, I used the postcode value from $_GET and passed it into location_get_postalcode_data() to get the lat/lon array. Then I passed both lat/lon arrays (the origin, and the one from $node->locations) into location_distance_between() and hey presto!

These little tricks enabled me to solve my problem and provide a working postal code search facility to my client, but I'm under no illusions about them being a solution flexible enough for anyone else, which is why I'm not providing comprehensive code, just guidelines to other stuck developers about how they might solve their problem. Hopefully a couple more iterations of Views or Location may solve these problems. I had to find a quick and clean-as-possible solution in order to meet a deadline. I found myself caught in the nether world where Views was able to provide so much of the required functionality that it became worth it to try to hack around in search of that remaining 5% rather than coding something from scratch.

None of the above should be interpreted as criticism of the developers who work on the Views and Location modules. These are incredibly complex modules which have to integrate and play nice with a number of other incredibly complex modules, and Drupal itself, so it's no wonder that there are bugs and omissions. It was actually quite exciting to find out that in the 2 weeks since I started development on my project, one of my critical bugs was fixed in a development release!

That concludes my unnecessarily long summary of a developer's solution to a client request. Hopefully someone might find it useful, and if you do, a comment would make me feel all warm and fuzzy!

EDIT #1:I realized I could make my life even easier by calculating latitude and longitude for the origin once (in the form submit handler) and then passing the latitude and longitude as values in the URL. This meant that instead of calling location_get_postalcode_data() 12 times (once in the form submit, once in the Views sort criteria, and once for each node teaser displayed in the View), it only gets called once. Much better.

EDIT #2: Another problem became apparent in testing: the UK postcode data that ships with the Location Module is a reduced set, containing data only up to the area (in the form 'CO1' or 'HG3') rather than the full set of postcode data ('CO1 xxx' or 'HG3 xxx'). When I searched for the full set of UK postcode data, it became apparent why this was: there were over 2 million rows of data to import.

For those interested, I found a Drupal-friendly SQL script for the full UK postcode data here (scroll down to comment #17). However, I found this to be a problem for me. Even once I got it imported, which took quite a while due to MySQL timeouts, I found it slowed my Search page down by a factor of 10 (from average 0.5 seconds to average 5 seconds), and almost all of that time was taken up by the MySQL query used in location_get_postalcode_data(). There didn't seem to be a lot I could do in terms of optimization so I decided to revert to the reduced set of data (which provides a fairly accurate location, it just doesn't go down to the street level and therefore some distance calculations are probably slightly off).

If anyone has a Drupal site, using Location, working with the full UK postcode data set, that isn't slow as a dog when searching that table, then I want to know!

EDIT #3: The client raised a problem that was interesting. After searching by postcode and displaying the View as a GMap, the map was always centred by default on the same location, which was set by a GMap macro in the View settings. The client wanted the map to be centred on the postcode that had just been searched for (a totally reasonable request). However, this wasn't an option. Views offered me the ability to center on a node argument; in other words, if I passed in a nodeid, I could center the map on that node's location. This wasn't very useful to me, since I want to center on the postcode argument, not a node.

One possible workaround occurred to me: on every postcode search, I can create a node, set the node's location to the searched postcode, and then pass that node id as an argument to the View. These dummy nodes could then be deleted on the next cron run. However, this kind of hack should be regarded as a last resort.

I ended up settling for a much more elegant hack which I discovered here. It involves adding 1 line of code to the GMap module which enables the GMap macro field in my view to parse PHP. Once this is done, then it's easy for me to insert some php that reads my postcode (or more accurately, my lat/lon arguments) and outputs the macro using those to center the map.

Of course, now I have to remember to re-insert this line in gmap.module every time I update the GMap module, which is why hacking modules is usually a bad idea. However in this case it was worth it to me, as a) a fix was needed quickly, b) this is obvious and necessary functionality, and c) I am not being retained as the site's maintainer ;-)

It's probably clear from all of the above that Drupal + Views + GMap + Location is almost, but not quite, ready for this extremely common real-world application (searching by postcode).

drupal

location

postcode

views

web development

click image to enlarge_

other recent blogs

Looking for a Drupal themer_

Drupal 6 Popup Forms_

Deleting cookies using PHP_

table row links in Drupal Views_

see all blog entries_

The New New Flesh

alan — Tue, 23 Mar 2010 03:11:30 +0000

my blog_The New New Flesh

23.03.2010

I'm nearly finished the new version of my own site - visuals provided by the fantastic Marc De'ath of the Creative Coop, who I've been working with for the past year and a half, back-end courtesy of Drupal, and sweat and late nights provided by me. Sometimes the simple-looking sites turn out to require the most work, or at least more than you expect!

This will be the first time I've ever had a personal (business) website designed professionally, as part of an overall branding exercise. When I say "branding", we're not going over the top with it. But I'll be having business cards and possibly letterheads printed, and I may even have an email signature, something that for some reason, through 10 years of working in IT, I have always managed to avoid.

Needless to say, I love the new design. I've discovered for myself over the last few years that my skills are definitely in the area of coding and development, rather than visual design. I doubt I could have made it on my own as a web designer/developer, but I've found that working together with graphic designers is both easier and more rewarding. I knew that it was a complicated skill to make something work right and do what it was supposed to do, but I didn't realize (until i tried it) that it is at least as complicated to make something look good, or even just to make it not look bad.

One thing I like about this new design is that it removes all the "fat" from my site. I always had a lot of features that I thought would be cool when I developed my own site - rotating quotations, photo albums, things like that. However, I found that life always keeps you busy, especially when you have children, and there just isn't time to keep updating those things, so after a while the same old images and text just sit there gathering virtual dust. This new design focuses me on the things that I do keep up with - blog entries, the odd software review, and my web portfolio. All the other stuff is elsewhere on the web - my photos are on Flickr, my random daily thoughts are on Twitter, and my writing is drifting in the ether like a poltergeist waiting to find a suitable host to terrorize.

new site