Joe Junkin

My Geocoding experience with Google and Yahoo became increasingly difficult and lengthy until I finally went looking for another solution. I found that solution with the Perl Module Geo::Coder::US. My final solution utilized Geo::Coder::US for most of the geocoding with Google and Yahoo helping on more difficult ones. The processing time for batch processing 20,000 real estate listings went from around 8 hours in the beginning down to about 1 minute in the end.

(more…)

The data provided by REIL does not contain geocoding which means that I will have to do this myself. I am using Perl to download and build the data so I have designed a Perl process to bulk geocode each of the addresses. I designed a caching system so that I only need to do any one lookup one time.

(more…)

I have been doing continued refinement on the reil rets download.  I decided to structure the download so that each ‘Class’ (residential, condo,etc) is downloaded into a separate table -as is. After the download is complete I run updates every 15 minutes. At first I attempted to place data into tables that would be accessed directly by the application, but I decided to change that process and create a final build table.

(more…)

My first activity was to download the property data. REIL has a bunch of property ‘classes’ such as RES (residential) and CON (Condominium). The classes will need to go into separate tables. So I designed my download process to take in a Resource (Property) and a Class (RES, CON, etc). Designing this way meant I could use the same routines for all classes of data.

(more…)

So, this is where I have ended up, writing the REIL data download from scratch. I am using Perl and imagine it would be difficult using something else. I want to be able to have one set of routines to download the data in whatever format I choose, and transfer the data to a MySQL database. Next I will need to download the images. I will need the data for all of the REIL property classes. Finally, I will need to have a routine that updates the data.

I spent a long time messing with VieleRETS to try and download data from REIL. The short answer is that I failed to get it to work. I tried to debug the PHP code but was unable to identify why it was failing. Had I needed to get this to work I would have joined the mail-list group and tried to get some help from there, but I never got that far.

(more…)

REIL.com uses digest authentication. I am using perl to pull down the data so first I need to login to the reil.com server with proper authorization. I was unable to find a lot of documentation on how this process works. The usual method for exposing web services is via SOAP, but this service uses HTTP Digest Authentication. I have not used it much but it appears that the main benefit is not sending the password in clear text over a non-ssl connection. It is not as secure as SSL by any means but is better than a clear text transmission of the password.
(more…)