search

estately / rets

A pure-ruby library for fetching data from RETS servers
https://github.com/estately/rets
127 stars 94 forks source link

Fetching images binary data from CRML #210

Closed gildardoperez closed 7 years ago

gildardoperez commented 7 years ago

Hi, I'm trying to get all of the photos downloaded using this example, however I get this error when trying to run the script.

ruby get-photos.rb

require 'rets'

client = Rets::Client.new({
  login_url: 'domainURL',
  username: 'username',
  password: 'password',
  version: 'RETS/1.7.2' 
})

begin
    client.login
rescue => e
    puts 'Error: ' + e.message
    exit!
end

puts 'We connected! Lets get all the photos for a property...'

photos = client.find (:first), {
  search_type: 'Media',
  class: 'Media',
  query: '(MediaType=Image)'

}

photos.each_with_index do |data, index|
  File.open("property-#{index.to_s}.jpg", 'w') do |file|
    file.write data.body
  end
end

puts photos.length.to_s + ' photos saved.'
client.logout
Cookie#domain returns dot-less domain name now. Use Cookie#dot_domain if you need "." at the beginning.
We connected! Lets get all the photos for a property...
current_index: 0
get-photos.rb:54:in `block (2 levels) in <main>': undefined method `body' for ["FileSize", ""]:Array (NoMethodError)
    from get-photos.rb:52:in `open'
    from get-photos.rb:52:in `block in <main>'
    from get-photos.rb:46:in `each'
    from get-photos.rb:46:in `each_with_index'
    from get-photos.rb:46:in `<main>'

any idea what that means? Thanks

dougcole commented 7 years ago

There is a specific method for fetching binary objects, take a look at the example here: https://github.com/estately/rets/blob/master/example/get-photos.rb

dougcole commented 7 years ago

short answer is use client.objects instead of client.find.

gildardoperez commented 7 years ago

tried using objects and got this error.

photos = client.objects '*', {
  resource: 'Media',
  object_type: 'Media',
  resource_id: 'MediaType=Image',
  limit: 10
}
/.rvm/gems/ruby-2.3.0@rails4.2/gems/rets-0.10.0/lib/rets/parser/error_checker.rb:33:in `check': Got error code 20401 (Invalid Type) (Rets::InvalidType)
    from /.rvm/gems/ruby-2.3.0@rails4.2/gems/rets-0.10.0/lib/rets/http_client.rb:73:in `http_post'
    from /.rvm/gems/ruby-2.3.0@rails4.2/gems/rets-0.10.0/lib/rets/client.rb:337:in `http_post'
    from /.rvm/gems/ruby-2.3.0@rails4.2/gems/rets-0.10.0/lib/rets/client.rb:240:in `fetch_object'
    from /.rvm/gems/ruby-2.3.0@rails4.2/gems/rets-0.10.0/lib/rets/client.rb:182:in `objects'
    from get-photos.rb:26:in `<main>'
dougcole commented 7 years ago

That's an error being returned from the server you're contacting them, I haven't seen their metadata so I'm not sure what the problem is, sorry. I'd ask the MLS about the error code.

gildardoperez commented 7 years ago

Hi, thanks for your help and time. Here is the output for the metadata corresponding to Media Media using the "client.find" method:

....
photos = client.find (:first), {
  search_type: 'Media',
  class: 'Media',
  query: '(MediaModificationTimestamp=2017-04-15+),(MediaType=Image)'
}

photos.each_with_index do |data, index|
  puts "#{data}"
end
...

We connected! Lets get all the photos for a property... ["FileSize", ""] ["Approved", ""] ["ResourceName", "1"] ["ImageOf", ""] ["ChangedByMemberID", ""] ["ChangedByMemberKeyNumeric", ""] ["ClassName", "2"] ["DeletedYN", "0"] ["ImageHeight", ""] ["ImageWidth", ""] ["LongDescription", ""] ["MediaHTML", ""] ["MediaKey", "237FF227-5FCD-450E-B1B0-63F28CD42969"] ["MediaModificationTimestamp", "2017-04-15 00:02:05"] ["MediaObjectID", "_K4_3268.jpg"] ["MediaURL", "http://media.crmls.org/medias/237ff227-5fcd-450e-b1b0-63f28cd42969.jpg"] ["MemberAOR", "SW"] ["MediaType", "IMAGE"] ["ModificationTimestamp", "2017-04-15 05:11:05"] ["OfficeMlsId", "ABA301"] ["Order", "0"] ["OriginatingSystemMediaKey", "253401934"] ["OriginatingSystemID", "MR"] ["ResourceRecordID", ""] ["ResourceRecordKeyNumeric", "112059030"] ["ShortDescription", ""] 26 photos saved.

gildardoperez commented 7 years ago

I think GetObject is not supported by CRMLS, so client.find works fine and gets photos but without any actual data.

....
photos = client.find (:first), {
  search_type: 'Media',
  class: 'Media',
  query: '(MediaModificationTimestamp=2017-04-15+),(MediaType=Image)'
}

# puts photos

photos.each_with_index do |data, index|
  File.open("property-#{index.to_s}.jpg", 'w') do |file|
    file.write data
  end
end
....
dougcole commented 7 years ago

ah, ok. I see it does return a MediaURL, where you can download the photos, that should work for you.

gildardoperez commented 7 years ago

👍 now my follow up question is how do I target MediaURL to file.write data? Since the above example renders this error.

["MediaURL", "http://media.crmls.org/medias/237ff227-5fcd-450e-b1b0-63f28cd42969.jpg"]

block (2 levels) in <main>': undefined method `body' for ["FileSize", ""]:Array (NoMethodError)

FileSize being the first array?

gildardoperez commented 7 years ago

Hi @dougcole I managed to get the data put for the MediaURL. However, I'm only getting one image instead of the 26 expected. Can you help?

require 'rets'

client = Rets::Client.new({
  login_url: 'url',
  username: 'user',
  password: 'password',
  version: 'RETS/1.7.2' 
})

begin
    client.login
rescue => e
    puts 'Error: ' + e.message
    exit!
end

puts 'We connected! Lets get all the photos for a property...'

photos = client.find (:first), {
  search_type: 'Media',
  class: 'Media',
  query: '(MediaModificationTimestamp=2017-04-15+),(MediaType=Image)'
}

photo = open(photo = photos['MediaURL'])
require 'base64'
image = Base64.encode64(photo.read)

File.open('property-1.gif', 'wb') do|f|
  f.write(Base64.decode64(image))
end

puts photos.length.to_s + ' photos saved.'
client.logout
dougcole commented 7 years ago

@gildardoperez you requested :first in your find query. If you want more photos you can use :all.

gildardoperez commented 7 years ago

Hi @dougcole. Thank you for the time. I'm attempting to first get the first listing to respond with each individual MediaURL corresponding to the listing. The "photos.length" shows 26 but the puts only gets the first image. Any idea why this might happen? 

....
photos = client.find (:first), {
  search_type: 'Media',
  class: 'Media',
  query: '(OriginatingSystemMediaKey=253852520),(MediaType=Image)'
}

photos.each_with_index do |data, index|
  puts photos['MediaURL']
end
...

This is my puts message, and I need to get each image corresponding to the OriginatingSystemMediaKey:

http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
http://media.crmls.org/medias/d2ef8208-a826-412d-91e5-c73bc5a2a1c4.jpg
26 photos saved.
dougcole commented 7 years ago

@gildardoperez did you try :all instead of :first? I'm not really sure what the problem is to be honest, but I think that's it. I'm also pretty confident it isn't a bug in the rets gem.

gildardoperez commented 7 years ago

My apologies, you are right. This isn't related to the RETS gem. Thanks.