thompsonbd.com :: Rets Example

Connecting to a RETS Server using PHP

I generally keep all static items that are reused in a config file. In the config file for the RETS connections I have put the server name and port. These you can get from the company managing your RETS server(Rapattoni). Your username, password, user-agent, and MLS ID should be provided by your association that handles the MLS.

The login_area and the login_uri will also be given to your by the company managing your RETS server(Rapattoni).

config.php


$server       = 'sub.domain.com';
$port         = 1111;
$username     = 'username';
$password     = 'password';
$mlsid        = MLSID;
$user_agent   = 'FDR/1.8';
$login_area   = 'GET /ASSOCIATIONNAME/MLSID/login.aspx?BrokerCode%3d HTTP/1.1';
$login_uri    = '/ASSOCIATIONNAME/MLSID/login.aspx';
$rets_version = 'RETS/1.5';
$qop          = 'auth';
$nc           = '00000001';
$nonce        = '';
$opaque       = '';

I also store a handful of functions in the config file.

RETS_CONNECT() is the function that handles the requests to the server and the response.


function RETS_CONNECT($server, $port, $out) {
    //  This ensures that all connections are closed.
    $out .= "Connection: Close\r\n\r\n";
    
    //  Creates socket connection to the RETS server
    $fp = fsockopen($server,$port,$errno,$errstr,30);
    
    //  IMPORTANT - This checks to make sure the socket connection is a valid socket connection

    //  If this wasn't done, and the connection wasn't a valid resource it can send the script into

    //  an endless loop and bomb the server

    if(!is_resource($fp)) {
        echo 'This connection is not a valid connection.<br />This script has been stopped to ensure your server does not crash.';die;exit;
    } else {
        echo '
            <div style="width: 100%;padding: 10px 0px;background-color: #999999;">
                Socket connection created to <strong>' . $server . '</strong> on port <strong>' . $port . '</strong>
            </div>
        ';
    }

    echo '<pre>' . $out . '</pre>';
    
    fputs($fp, $out);
    
    while (!feof($fp)) {
        $response .= fgets($fp, 128);
    }
    
    fclose($fp);

    echo '<pre>' . $response . '</pre>';

    return $response;
}

This function is used to strip items from the server response that we need for the next step.


//  Used to strip nonce, auth, opaque, and session id

function PARSE_RESPONSE($haystack, $needle){

    return substr(strstr($haystack, $needle),(strlen($needle) + 1),32);
}

The following example is an example that connects to the server, switches to the search page, and retrieves all residential and land listings, along with their photos. It takes the returned information and places it in a MySQL database. You will need to change a few items such as "ASSOCIATIONNAME" and "MLSID".

update.php


<?
require('config.php');

//  A1 is used in the response

$A1 = md5($username . ':' . $mlsid . ':' . $password);

//  Not sure yet what this does or is used for.  I believe it is the request method and location
//  (method:digest-uri-value)
$A2 = md5('GET:' . $login_uri);

//  Response to the server.  The A1 and A2 have already been md5'd before this.
//  md5(md5(A1):nonce:md5(A2))
$resp = md5($A1 . ':' . $nonce . ':' . $A2);<

//  Request connection
$out = $login_area . "\r\n";
$out .= 'RETS-Version: ' . $rets_version . "\r\n";
$out .= 'Authorization: Digest username="' . $username . '", realm="' . $mlsid . '", nonce="' . $nonce . '", opaque="' . $opaque . '", uri="' . $login_uri . '", response="' . $resp . '"' . " \r\n";
$out .= 'Host: ' . $server . ':' . $port . "\r\n";
$out .= "Accept: */*\r\n";
$out .= 'User-Agent: ' . $user_agent . "\r\n";

$response_initial = RETS_CONNECT($server, $port, $out);

//  Parse initial server response and get nonce and opaque from it.
$nonce = PARSE_RESPONSE($response_initial, 'nonce=');
$opaque = PARSE_RESPONSE($response_initial, 'opaque=');

$cnonce = md5($user_agent . ':::' . $nonce);

$resp = md5($A1 . ':' . $nonce . ':' . $nc . ':' . $cnonce . ':' . $qop . ':' . $A2);

//  Auth and get session id
$out = '';
$out = $login_area . "\r\n";
$out .= 'RETS-Version: ' . $rets_version . "\r\n";
$out .= 'Authorization: Digest username="' . $username . '", realm="' . $mlsid . '", nonce="' . $nonce . '", uri="' . $login_uri . '", cnonce="' . $cnonce . '", nc=' . $nc . ', qop="' . $qop . '", response="' . $resp . '", opaque="' . $opaque . '"' . " \r\n";
$out .= 'Host: ' . $server . ':' . $port . "\r\n";
$out .= "Accept: */*\r\n";
$out .= 'User-Agent: ' . $user_agent . "\r\n";

$response_authed = RETS_CONNECT($server, $port, $out);

$session_id = PARSE_RESPONSE($response_authed, 'RETS-Session-ID');  //  LEAVE THE = OUT

//  Switch to search and request auth
$out = '';
$out = "GET /ASSOCIATIONNAME/MLSID/search.aspx HTTP/1.1\r\n";
$out .= 'RETS-Version: ' . $rets_version . "\r\n";
$out .= 'Host: ' . $server . ':' . $port . "\r\n";
$out .= 'Cookie: RETS-Session-ID=' . $session_id . "\r\n";
$out .= "Accept: */*\r\n";
$out .= 'User-Agent: ' . $user_agent . "\r\n";

$response_initial = RETS_CONNECT($server, $port, $out);

$nonce = PARSE_RESPONSE($response_initial, 'nonce=');
$opaque = PARSE_RESPONSE($response_initial, 'opaque=');

$A2 = md5('GET:/ASSOCIATIONNAME/MLSID/search.aspx');  //  (method:digest-uri-value)
$resp = md5($A1 . ':' . $nonce . ':' . $A2);

$cnonce = md5($user_agent . ':::' . $nonce);

$resp = md5($A1 . ':' . $nonce . ':' . $nc . ':' . $cnonce . ':' . $qop . ':' . $A2);

In the next step we will actually build our query for retrieving the listings we want.

Class=RESI is used for residential listings.

Select = ................. are the actual fields we want returned.

Query=(ListingOfficeMLSID=############),(MLNumber=1+) gets only listings from the office I want and MLNumber=1+ is used to trick the server into thinking we have submited 2 limiting fields. This is only needed on some servers.


//  authenicate in search and submit residential query

$query = rawurlencode('Class=RESI&Select=MLNumber,Status,ListingPrice,StreetNumber,StreetName,

StreetSuffix,City,State,ZipCode,Subdivision,Area,RESIPROP,RESIWATE,RESIWTRT,Bedrooms,

FullBathrooms,HalfBathrooms,SquareFootage,YearBuilt,Longitude,Latitude,

ListingAgentFirstName,ListingAgentLastName,RESILTFR,RESILTLT,RESILTRT,RESILTRR,

RESIANND,RESIAPPL,RESIDIND,RESIEXTF,RESIFIRE,RESIFLOR,RESIINTR,RESIWTRS,

MarketingRemarks&Query=(ListingOfficeMLSID=############),(MLNumber=1+)

&SearchType=Property&Format=COMPACT-DECODED&QueryType=DMQL2&Limit=NONE

&Count=0');



$out = 'GET /ASSOCIATIONNAME/MLSID/search.aspx?' . $query . " HTTP/1.1\r\n";

$out .= 'RETS-Version: ' . $rets_version . "\r\n";

$out .= 'Authorization: Digest username="' . $username . '", realm="' . $mlsid . '", nonce="' . $nonce . '", uri="/ASSOCIATIONNAME/MLSID/search.aspx", cnonce="' . $cnonce . '", nc=' . $nc . ', qop="' . $qop . '", response="' . $resp . '", opaque="' . $opaque . '"' . " \r\n";

$out .= 'Host: ' . $server . ':' . $port . "\r\n";

$out .= 'Cookie: RETS-Session-ID=' . $session_id . "\r\n";

$out .= "Accept: */*\r\n";

$out .= 'User-Agent: ' . $user_agent . "\r\n";



$response = RETS_CONNECT($server, $port, $out);



$prop_array = explode("\t", $response);



$fields = 36;  //  Same as the number of fields requested in $query

$start = $fields + 2;



echo '<table>';

while($prop_array[$start] > 40000) {



$mls = $prop_array[$start++];

$status = $prop_array[$start++];

$price = substr($prop_array[$start++],0,-3);

$address = $prop_array[$start++] . ' ' . $prop_array[$start++] . ' ' . $prop_array[$start++];

$city = $prop_array[$start++];

$state = $prop_array[$start++];

$zip = $prop_array[$start++];

$subdivision = $prop_array[$start++];

$area = $prop_array[$start++];

$location = $prop_array[$start++];

$waterfront = $prop_array[$start++];

if($waterfront == 'Yes'){

$location = $prop_array[$start++];

}

else {

$waterfront_type = $prop_array[$start++];

}

$bedrooms =  substr($prop_array[$start++],0,1);

$baths = $prop_array[$start++];

$halfbath = $prop_array[$start++];

$sqft = $prop_array[$start++];

$yearbuilt = $prop_array[$start++];

$longitude = $prop_array[$start++];

$latitude = $prop_array[$start++];

$lister_fname = $prop_array[$start++];

$lister_lname = $prop_array[$start++];

$lot_front = $prop_array[$start++];

$lot_left = $prop_array[$start++];

$lot_right = $prop_array[$start++];

$lot_rear = $prop_array[$start++];

$assoc_dues = $prop_array[$start++];

$appliances = $prop_array[$start++];

$dining_desc = $prop_array[$start++];

$ext_features = $prop_array[$start++];

$fireplaces = $prop_array[$start++];

$floors = $prop_array[$start++];

$int_features = $prop_array[$start++];

$water_sewer = $prop_array[$start++];

$desc = addslashes($prop_array[$start++]);



$date = date("Y-m-d G:i:s");



$sql_insert = "INSERT INTO properties

(prop_mls, prop_status, prop_price, prop_address, prop_city, prop_state, prop_zip, prop_subdivision, prop_area, prop_location, prop_waterfront_type, prop_bedrooms, prop_baths, prop_halfbaths, prop_sqft, prop_yearbuilt, prop_longitude, prop_latitude, prop_lister_fname,

prop_lister_lname, prop_desc,prop_updated,prop_lot_left,prop_lot_right,prop_lot_front,

prop_lot_rear,prop_appliances,prop_assoc_due,prop_dining_desc,

prop_ext_features,prop_fireplaces,prop_floors,
prop_int_features,prop_sewer_water)

VALUES ('" . $mls . "','" . $status . "','" . $price . "','" . $address . "','" . $city . "','" . $state . "','" . $zip . "',

'" . $subdivision . "','" . $area . "','" . $location . "','" . $waterfront_type . "','" . $bedrooms . "','" . $baths . "',

'" . $halfbaths . "','" . $sqft . "','" . $yearbuilt . "','" . $longitude . "','" . $latitude . "','" . $lister_fname . "','" . $lister_lname . "','" . $desc . "','$date','" . $lot_left . "','" . $lot_right . "','" . $lot_front . "','" . $lot_rear . "','" . $appliances . "','" . $assoc_dues . "','" . $dining_desc . "','" . $ext_features . "','" . $fireplaces . "','" . $floors . "','" . $int_features . "','" . $sewer_water . "');";



$rs_insert = mysql_query($sql_insert);



echo '<tr>';

if($rs_insert) { 

    echo '<td style="border-bottom:1px solid #000000;border-top:1px solid #000000;border-left:1px solid #000000;background-color:#999999;">inserted</td>';

    echo '<td style="border-bottom:1px solid #000000;border-top:1px solid #000000;border-right:1px solid #000000;background-color:#999999;">' . $sql_insert . '</td>';

}

else{

    $sql_update = "UPDATE properties

    SET prop_status = '$status', prop_price = '$price', prop_address = '$address', prop_city = '$city', prop_state = '$state', prop_zip = '$zip', prop_subdivision = '$subdivision', prop_area = '$area', prop_location = '$location', prop_bedrooms = '$bedrooms', prop_baths = '$baths', prop_halfbaths = '$halfbaths', prop_sqft = '$sqft', prop_yearbuilt = '$yearbuilt', prop_longitude = '$longitude', prop_latitude = '$latitude', prop_lister_fname = '$lister_fname', prop_lister_lname = '$lister_lname', prop_desc = '$desc', prop_updated = '" . date("Y-m-d G:i:s") . "', prop_lot_left = '$lot_left', prop_lot_right = '$lot_right', prop_lot_front = '$lot_front', prop_lot_rear = '$lot_rear', prop_appliances = '$appliances', prop_assoc_due = '$assoc_dues', prop_dining_desc = '$dining_desc', prop_ext_features = '$ext_features', prop_fireplaces = '$fireplaces',prop_floors = '$floors', prop_int_features = '$int_features', prop_sewer_water = '$sewer_water', prop_updated = '$date'

    WHERE prop_mls = '$mls';";

    $rs_update = mysql_query($sql_update);

    if($rs_update) {

        echo '<td style="border-bottom:1px solid #000000;border-top:1px solid #000000;border-left:1px solid #000000;background-color:#999999;">updated</td>';

    }

    else{

        echo '<td style="border-bottom:1px solid #000000;border-top:1px solid #000000;border-left:1px solid #000000;background-color:#999999;">FAILED</td>';

    }

    echo '<td style="border-bottom:1px solid #000000;border-top:1px solid #000000;border-right:1px solid #000000;background-color:#999999;">' . $sql_update . '</td>';

}

echo '</tr>';

echo '<tr><td colspan="2"><table><tr>';



//original image

$img = 'http://sub.domain.com/MLSID/listingpics/bigphoto/0' . substr($mls, -2, 2) . '/' . $mls . '.jpg';



//directory to copy to (must be CHMOD to 777)

$copydir = '/home/httpd/vhosts/mydomain.com/httpdocs/property_images/';



$data = file_get_contents($img); 

if($data){

    $file = fopen($copydir . $mls . '.jpg', "w+"); 

    if(fputs($file, $data)){

        echo '<td>' . $mls . '</td>';

    } 

    fclose($file);

} 



for($i=1;$i<20;$i++) {

    if($i < 10){

        $image = $mls . "_0" . $i;

    }

    else {

        $image = $mls . '_' . $i;

    }

//original image

    $img = 'http://sub.domain.com/MLSID/listingpics/bigphoto/0' . substr($mls, -2, 2) . '/' . $image . '.jpg';



//directory to copy to (must be CHMOD to 777)

    $copydir = '/home/httpd/vhosts/mydomain.com/httpdocs/property_images/';

    $data = file_get_contents($img); 

    if($data){

        $file = fopen($copydir . $image . '.jpg', "w+"); 

        if(fputs($file, $data)){

            echo '<td>' . $image . '</td>';

        } 

        fclose($file);

        } 

    } 

    echo '</tr></td></table></tr>';



    $start++;

}

echo '</table>'; 



echo '<div align="center">UPDATE DONE</div>';

The next piece of code removes all listings that were not updated and deletes their pictures.


$sql = "SELECT prop_mls FROM properties WHERE prop_updated < '$date';";

$rs = mysql_query($sql);



while($row = mysql_fetch_object($rs)) {

    $sql = "DELETE FROM properties WHERE prop_mls = '" . $row->prop_mls . "';";

    if($rs = mysql_query($sql)){

        echo 'deleted ' . $row->prop_mls . '<br />';

    }



//directory to copy to (must be CHMOD to 777)

    $copydir = '/home/httpd/vhosts/holdenbeach.com/httpdocs/property_images/';



    $file = $copydir . $row->prop_mls . '.jpg';

    if(file_exists($file)){

        unlink($file);

    } 



    for($i=1;$i<20;$i++) {

        if($i < 10){

            $image = $row->prop_mls . "_0" . $i;

        }

        else {

            $image = $row->prop_mls . '_' . $i;

        }



        $file = $copydir . $image . '.jpg';

        if(file_exists($file)){

            unlink($file);

        } 

    } 

}



?>