RETs, librets, and Perl

[tempire]

created_at=2009-11-12T19:46:45

Of all the protocols I've had to learn, RETS, (Real Estate Transaction Standard) is the worst.  Maybe it's because the entire real estate industry thrives on secrecy.  Maybe it's because there just aren't that many people working with real estate data.  Maybe it's because the Fort Lauderdale Real Estate IT department has no interest in helping anyone not working on Windows.

 

Four points worth knowing

If you're a RETS n00b, these four points of information will save you quite a bit of time:

• RETS is a psuedo XML-based loose standard that allows retreival of MLS data. Psuedo, meaning there are blobs of space-delimited data within xml blocks
• Features of each property are identified via numbers called System names. The system name for the feature "List Price", for example, might be 137. Standard Names are where you would find the "List Price" name, although some servers to not support this. An informal description might also be identified as a Long Name.
• Most RETS servers limit your results, so retrieving all listings and storing them locally is a potential problem.  You might have to retrieve the data in chunks, such as by zip code or area. 
• The unique for each listing might not be the MLS#, as you would expect.  It could be, though for MarketLinx servers, the "sysid" is the unique id. 

Fortunately, librets provides a relatively easy API in C++, and via Swig: Perl, Python, PHP, Java, C#, and Ruby.  They provide some very helpful examples that work with their demo server, but not in my case. 

My examples work with a active MarketLinx server, based on RETS 1.72, as is necessary when working with Fort Lauderdale listings.  MarketLinx supports only standard names, so working with the metadata is a must.

To install librets on OSX 10.5 or 10.6:

1. Install Macports
2. Install dependencies

   sudo port install boost swig

3. Download & install librets

   curl -O http://www.crt.realtors.org/projects/rets/librets/files/librets-1.3.3.tar.gz
   tar xzf librets-1.3.3.tar.gz
   cd librets-1.3.3
   ./configure
   make
   make install
   

   I didn't want to install everything, so I left off the make install above. To install only the perl library:

   cd build/swig/perl
   perl Makefile.PL
   make
   make install
   

You should now have a working librets installation. 

To search for properties:

#!/usr/bin/env perl
use lib "blib/lib", "blib/arch"; 
use strict; 
use librets;

my $rets = new librets::RetsSession(
    "http://ragfl.retsiq.com/contact/rets/login"
);

$rets->SetUserAgentAuthType(
   $librets::UserAgentAuthType::USER_AGENT_AUTH_RETS_1_7
);

if (!$rets->Login("USERNAME", "PASSWORD"))
{
    print "Invalid login\n";
    exit 1;
}

# Very useful for determining differences between RETS servers
# and understanding the metadata
$rets->SetHttpLogName("log");

# Get metadata
my $metadata = $rets->GetMetadata;

# Get classes, and put all properties features into hashref, 
# with system names as keys
# ex: { 137 => "List Price", "19" => "Bedrooms" }
my $class = $metadata->GetClass( "Property", "1" );
my $tables = $metadata->GetAllTables( $class );
my %features;
foreach my $table ( @$tables ) {
   $features{ $table->GetSystemName } = $table->GetLongName;
}

# Search for MLS Listing F1027824
my $request = $rets->CreateSearchRequest( 
   "Property", "1", "(157=F1027824)" 
);

# Search for MLS Listing F1020131 or F1027824
# my $request = $rets->CreateSearchRequest( 
#    "Property", "1", "(157=F1020131)|(157=F1027824)"
# );

# Search for MLS Listings having a List price between 300k and 310k
# my $request = $rets->CreateSearchRequest( 
#    "Property", "1", "(137=300000-310000)"
# );

# Turn off standard names - necessary if server does not support it
$request->SetStandardNames(0);

# SetSelect accepts system names (number codes) as parameters, 
# to only return certain property features
# By default, all features are returned
# $request->SetSelect("137,19");

$request->SetLimit( $librets::SearchRequest::LIMIT_DEFAULT );

# Offset is optional in the RETS spec, unfortunately.  
# Marketlinx does not support it :(
$request->SetOffset( $librets::SearchRequest::OFFSET_NONE );

$request->SetCountType( 
   $librets::SearchRequest::RECORD_COUNT_AND_RESULTS
);

my $results = $rets->Search( $request );
print "Record count: " . $results->GetCount() . "\n\n";

# All features for results (system names)
my $columns = $results->GetColumns();

# Loop through each property
while ($results->HasNext())
{
    foreach my $column (@$columns)
    {
       # Print each feature with code, name, and value
       print "($column) $features{$column}: " . 
          $results->GetString($column) . "\n";
    }
    print "\n";
}

my $logout =  $rets->Logout();

Running this script will give you output similar to the following:

Record count: 1

(1) Property Type: Single Family
(10) Zip Code: 33312
(100) Additional Furnished Info.: 
(101) Furnished Info (Sold): 
(102) #Garage Spaces: 0
(103) Garage Description: 
...

[etewiah]

4 years later and RETS is still a pain to work with but still important in the Real Estate sector!

[clipartinc]

11 years later and RETS is still a pain to work with in my local Real Estate division.

[tempire]

Dear goodness was just arbitrary content I uploaded to test something else - not even the original blog entry. I can't believe someone found it via the googles.

I hope it helps.

But yes. RETs is awful. I am sorry for all that had a need for this at all.