Response doesn't return JSON

I tried executing a basic query for NY ex-pats (JSON below) using utils.process_audience_from_response but the query was unable to complete because the response did not contain a JSON. Do you know if the API has changed and is now returning non-JSON responses?

query:

{   "name": "Expats Distribution in New York",
    "geo_locations": [
      { 
        "name": "cities",
        "values": [{"key":"2490299"}],
        "location_types": ["home"] 
      }
    ],
    "behavior": [
        {"or" : [6019673233983], "name" : "Expats (Zimbabwe)"},
        {"or" : [6027149006383], "name" : "Expats (Vietnam)"},
        {"or" : [6026404871583], "name" : "Expats (Venezuela)"},
        {"or" : [6023620475783], "name" : "Expats (US)"},
        {"or" : [6019396649183], "name" : "Expats (United States)"},
        {"or" : [6021354152983], "name" : "Expats (UK)"},
        {"or" : [6019673501783], "name" : "Expats (Uganda)"},
        {"or" : [6023516430783], "name" : "Expats (UAE)"},
        {"or" : [6019366994183], "name" : "Expats (Turkey)"},
        {"or" : [6019377644783], "name" : "Expats (Switzerland)"},
        {"or" : [6023516315983], "name" : "Expats (Sri Lanka)"},
        {"or" : [6019366943583], "name" : "Expats (Spain)"},
        {"or" : [6027148973583], "name" : "Expats (South Korea)"},
        {"or" : [6019564383383], "name" : "Expats (South Africa)"},
        {"or" : [6023516403783], "name" : "Expats (Singapore)"},
        {"or" : [6027149004983], "name" : "Expats (Serbia)"},
        {"or" : [6023357000583], "name" : "Expats (Senegal)"},
        {"or" : [6025000813183], "name" : "Expats (Saudi Arabia)"},
        {"or" : [6025670492783], "name" : "Expats (Rwanda)"},
        {"or" : [6025000815983], "name" : "Expats (Russia)"},
        {"or" : [6027148962983], "name" : "Expats (Romania)"},
        {"or" : [6019520122583], "name" : "Expats (Puerto Rico)"},
        {"or" : [6021354882783], "name" : "Expats (Portugal)"},
        {"or" : [6019396657183], "name" : "Expats (Poland)"},
        {"or" : [6018797091183], "name" : "Expats (Philippines)"},
        {"or" : [6027149008183], "name" : "Expats (Peru)"},
        {"or" : [6016916394383], "name" : "Expats (Pakistani)"},
        {"or" : [6027147189983], "name" : "Expats (Pakistan)"},
        {"or" : [6018797004183], "name" : "Expats (Nigeria)"},
        {"or" : [6023516368383], "name" : "Expats (New Zealand)"},
        {"or" : [6023287393783], "name" : "Expats (Netherlands)"},
        {"or" : [6023356955383], "name" : "Expats (Nepal)"},
        {"or" : [6023516338783], "name" : "Expats (Morocco)"},
        {"or" : [6023676072183], "name" : "Expats (Mexico)"},
        {"or" : [6027147160983], "name" : "Expats (Malaysia)"},
        {"or" : [6068613839383], "name" : "Expats (Latvia)"},
        {"or" : [6018796980983], "name" : "Expats (Kenya)"},
        {"or" : [6023676028783], "name" : "Expats (Japan)"},
        {"or" : [6019396654583], "name" : "Expats (Italy)"},
        {"or" : [6025000823583], "name" : "Expats (Israel)"},
        {"or" : [6019396650783], "name" : "Expats (Ireland)"},
        {"or" : [6027149017383], "name" : "Expats (Iran)"},
        {"or" : [6019564344583], "name" : "Expats (Indonesia)"},
        {"or" : [6016916298983], "name" : "Expats (India)"},
        {"or" : [6019396638383], "name" : "Expats (Hungary)"},
        {"or" : [6023676022783], "name" : "Expats (Hong Kong)"},
        {"or" : [6059793664583], "name" : "Expats (Honduras)"},
        {"or" : [6018797373783], "name" : "Expats (Haiti)"},
        {"or" : [6019673808383], "name" : "Expats (Guatemala)"},
        {"or" : [6023676017583], "name" : "Expats (Greece)"},
        {"or" : [6019673448383], "name" : "Expats (Ghana)"},
        {"or" : [6019367052983], "name" : "Expats (Germany)"},
        {"or" : [6019367014383], "name" : "Expats (France)"},
        {"or" : [6068209522983], "name" : "Expats (Finland)"},
        {"or" : [6018797165983], "name" : "Expats (Ethiopia)"},
        {"or" : [6023287351383], "name" : "Expats (Estonian)"},
        {"or" : [6019673777983], "name" : "Expats (El Salvador)"},
        {"or" : [6027148953383], "name" : "Expats (Egypt)"},
        {"or" : [6019673762183], "name" : "Expats (Dominican Republic)"},
        {"or" : [6018797127383], "name" : "Expats (Cuba)"},
        {"or" : [6019673525983], "name" : "Expats (Colombia)"},
        {"or" : [6019452369983], "name" : "Expats (China)"},
        {"or" : [6025054896983], "name" : "Expats (Chile)"},
        {"or" : [6019396764183], "name" : "Expats (Canada)"},
        {"or" : [6018797036783], "name" : "Expats (Cameroon)"},
        {"or" : [6019564340583], "name" : "Expats (Brazil)"},
        {"or" : [6043702804583], "name" : "Expats (Belgium)"},
        {"or" : [6023356562783], "name" : "Expats (Bangladesh)"},
        {"or" : [6023675997383], "name" : "Expats (Austria)"},
        {"or" : [6021354857783], "name" : "Expats (Australia)"},
        {"or" : [6025000826583], "name" : "Expats (Argentina)"}
    ],
    "ages_ranges": [
        {"min":18}
    ],
    "genders": [0]
}

error:

ValueErrorTraceback (most recent call last)
<ipython-input-24-33f40b778386> in <module>()
     25 
     26 watcher.load_credentials_direct(FB_KEY,FB_APPLICATION_ID)
---> 27 watcher.run_data_collection(SETTINGS_FILE_NAME)
     28 files.download('results.csv')

/usr/local/lib/python2.7/dist-packages/pysocialwatcher/main.pyc in run_data_collection(json_input_file_path)
    177         PySocialWatcher.check_input_integrity(input_data_json)
    178         collection_dataframe = PySocialWatcher.build_collection_dataframe(input_data_json)
--> 179         collection_dataframe = PySocialWatcher.perform_collection_data_on_facebook(collection_dataframe)
    180         return collection_dataframe
    181 

/usr/local/lib/python2.7/dist-packages/pysocialwatcher/main.pyc in perform_collection_data_on_facebook(collection_dataframe)
    142         print_info("Data Collection Complete")
    143         save_temporary_dataframe(collection_dataframe)
--> 144         post_process_collection(collection_dataframe)
    145         save_after_collecting_dataframe(collection_dataframe)
    146         return collection_dataframe

/usr/local/lib/python2.7/dist-packages/pysocialwatcher/utils.pyc in post_process_collection(collection_dataframe)
    327     print_info("Computing Audience column")
    328     collection_dataframe["audience"] = collection_dataframe["response"].apply(
--> 329         lambda x: process_audience_from_response(x))
    330     return collection_dataframe
    331 

/usr/local/lib/python2.7/dist-packages/pandas/core/series.pyc in apply(self, func, convert_dtype, args, **kwds)
   2549         index_col : int or sequence, default 0
   2550             Column to use for index. If a sequence is given, a MultiIndex
-> 2551             is used. Different default from read_table
   2552         encoding : string, optional
   2553             a string representing the encoding to use if the contents are

pandas/_libs/src/inference.pyx in pandas._libs.lib.map_infer()

/usr/local/lib/python2.7/dist-packages/pysocialwatcher/utils.pyc in <lambda>(x)
    327     print_info("Computing Audience column")
    328     collection_dataframe["audience"] = collection_dataframe["response"].apply(
--> 329         lambda x: process_audience_from_response(x))
    330     return collection_dataframe
    331 

/usr/local/lib/python2.7/dist-packages/pysocialwatcher/utils.pyc in process_audience_from_response(literal_response)
    319 
    320 def process_audience_from_response(literal_response):
--> 321     audience = json.loads(literal_response)["data"]["users"]
    322     return int(audience)
    323 

/usr/lib/python2.7/json/__init__.pyc in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    337             parse_int is None and parse_float is None and
    338             parse_constant is None and object_pairs_hook is None and not kw):
--> 339         return _default_decoder.decode(s)
    340     if cls is None:
    341         cls = JSONDecoder

/usr/lib/python2.7/json/decoder.pyc in decode(self, s, _w)
    362 
    363         """
--> 364         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    365         end = _w(s, end).end()
    366         if end != len(s):

/usr/lib/python2.7/json/decoder.pyc in raw_decode(self, s, idx)
    380             obj, end = self.scan_once(s, idx)
    381         except StopIteration:
--> 382             raise ValueError("No JSON object could be decoded")
    383         return obj, end

ValueError: No JSON object could be decoded

It seems that this happened. I would check each response in the temporary file created by pySocialWatcher after collecting the data to find what happened.

Matheus Araújo M.Sc in Computer Science Pursuing Ph.D. at the University of Minnesota.

On Sat, May 12, 2018 at 5:15 PM, Ian notifications@github.com wrote:

I tried executing a basic query for NY ex-pats (JSON below) using utils.process_audience_from_response but the query was unable to complete because the response did not contain a JSON. Do you know if the API has changed and is now returning non-JSON responses?

query:

{ "name": "Expats Distribution in New York", "geo_locations": [ { "name": "cities", "values": [{"key":"2490299"}], "location_types": ["home"] } ], "behavior": [ {"or" : [6019673233983], "name" : "Expats (Zimbabwe)"}, {"or" : [6027149006383], "name" : "Expats (Vietnam)"}, {"or" : [6026404871583], "name" : "Expats (Venezuela)"}, {"or" : [6023620475783], "name" : "Expats (US)"}, {"or" : [6019396649183], "name" : "Expats (United States)"}, {"or" : [6021354152983], "name" : "Expats (UK)"}, {"or" : [6019673501783], "name" : "Expats (Uganda)"}, {"or" : [6023516430783], "name" : "Expats (UAE)"}, {"or" : [6019366994183], "name" : "Expats (Turkey)"}, {"or" : [6019377644783], "name" : "Expats (Switzerland)"}, {"or" : [6023516315983], "name" : "Expats (Sri Lanka)"}, {"or" : [6019366943583], "name" : "Expats (Spain)"}, {"or" : [6027148973583], "name" : "Expats (South Korea)"}, {"or" : [6019564383383], "name" : "Expats (South Africa)"}, {"or" : [6023516403783], "name" : "Expats (Singapore)"}, {"or" : [6027149004983], "name" : "Expats (Serbia)"}, {"or" : [6023357000583], "name" : "Expats (Senegal)"}, {"or" : [6025000813183], "name" : "Expats (Saudi Arabia)"}, {"or" : [6025670492783], "name" : "Expats (Rwanda)"}, {"or" : [6025000815983], "name" : "Expats (Russia)"}, {"or" : [6027148962983], "name" : "Expats (Romania)"}, {"or" : [6019520122583], "name" : "Expats (Puerto Rico)"}, {"or" : [6021354882783], "name" : "Expats (Portugal)"}, {"or" : [6019396657183], "name" : "Expats (Poland)"}, {"or" : [6018797091183], "name" : "Expats (Philippines)"}, {"or" : [6027149008183], "name" : "Expats (Peru)"}, {"or" : [6016916394383], "name" : "Expats (Pakistani)"}, {"or" : [6027147189983], "name" : "Expats (Pakistan)"}, {"or" : [6018797004183], "name" : "Expats (Nigeria)"}, {"or" : [6023516368383], "name" : "Expats (New Zealand)"}, {"or" : [6023287393783], "name" : "Expats (Netherlands)"}, {"or" : [6023356955383], "name" : "Expats (Nepal)"}, {"or" : [6023516338783], "name" : "Expats (Morocco)"}, {"or" : [6023676072183], "name" : "Expats (Mexico)"}, {"or" : [6027147160983], "name" : "Expats (Malaysia)"}, {"or" : [6068613839383], "name" : "Expats (Latvia)"}, {"or" : [6018796980983], "name" : "Expats (Kenya)"}, {"or" : [6023676028783], "name" : "Expats (Japan)"}, {"or" : [6019396654583], "name" : "Expats (Italy)"}, {"or" : [6025000823583], "name" : "Expats (Israel)"}, {"or" : [6019396650783], "name" : "Expats (Ireland)"}, {"or" : [6027149017383], "name" : "Expats (Iran)"}, {"or" : [6019564344583], "name" : "Expats (Indonesia)"}, {"or" : [6016916298983], "name" : "Expats (India)"}, {"or" : [6019396638383], "name" : "Expats (Hungary)"}, {"or" : [6023676022783], "name" : "Expats (Hong Kong)"}, {"or" : [6059793664583], "name" : "Expats (Honduras)"}, {"or" : [6018797373783], "name" : "Expats (Haiti)"}, {"or" : [6019673808383], "name" : "Expats (Guatemala)"}, {"or" : [6023676017583], "name" : "Expats (Greece)"}, {"or" : [6019673448383], "name" : "Expats (Ghana)"}, {"or" : [6019367052983], "name" : "Expats (Germany)"}, {"or" : [6019367014383], "name" : "Expats (France)"}, {"or" : [6068209522983], "name" : "Expats (Finland)"}, {"or" : [6018797165983], "name" : "Expats (Ethiopia)"}, {"or" : [6023287351383], "name" : "Expats (Estonian)"}, {"or" : [6019673777983], "name" : "Expats (El Salvador)"}, {"or" : [6027148953383], "name" : "Expats (Egypt)"}, {"or" : [6019673762183], "name" : "Expats (Dominican Republic)"}, {"or" : [6018797127383], "name" : "Expats (Cuba)"}, {"or" : [6019673525983], "name" : "Expats (Colombia)"}, {"or" : [6019452369983], "name" : "Expats (China)"}, {"or" : [6025054896983], "name" : "Expats (Chile)"}, {"or" : [6019396764183], "name" : "Expats (Canada)"}, {"or" : [6018797036783], "name" : "Expats (Cameroon)"}, {"or" : [6019564340583], "name" : "Expats (Brazil)"}, {"or" : [6043702804583], "name" : "Expats (Belgium)"}, {"or" : [6023356562783], "name" : "Expats (Bangladesh)"}, {"or" : [6023675997383], "name" : "Expats (Austria)"}, {"or" : [6021354857783], "name" : "Expats (Australia)"}, {"or" : [6025000826583], "name" : "Expats (Argentina)"} ], "ages_ranges": [ {"min":18} ], "genders": [0] }

error:

ValueErrorTraceback (most recent call last)
in () 25 26 watcher.load_credentials_direct(FB_KEY,FB_APPLICATION_ID) ---> 27 watcher.run_data_collection(SETTINGS_FILE_NAME) 28 files.download('results.csv') /usr/local/lib/python2.7/dist-packages/pysocialwatcher/main.pyc in run_data_collection(json_input_file_path) 177 PySocialWatcher.check_input_integrity(input_data_json) 178 collection_dataframe = PySocialWatcher.build_collection_dataframe(input_data_json) --> 179 collection_dataframe = PySocialWatcher.perform_collection_data_on_facebook(collection_dataframe) 180 return collection_dataframe 181 /usr/local/lib/python2.7/dist-packages/pysocialwatcher/main.pyc in perform_collection_data_on_facebook(collection_dataframe) 142 print_info("Data Collection Complete") 143 save_temporary_dataframe(collection_dataframe) --> 144 post_process_collection(collection_dataframe) 145 save_after_collecting_dataframe(collection_dataframe) 146 return collection_dataframe /usr/local/lib/python2.7/dist-packages/pysocialwatcher/utils.pyc in post_process_collection(collection_dataframe) 327 print_info("Computing Audience column") 328 collection_dataframe["audience"] = collection_dataframe["response"].apply( --> 329 lambda x: process_audience_from_response(x)) 330 return collection_dataframe 331 /usr/local/lib/python2.7/dist-packages/pandas/core/series.pyc in apply(self, func, convert_dtype, args, **kwds) 2549 index_col : int or sequence, default 0 2550 Column to use for index. If a sequence is given, a MultiIndex -> 2551 is used. Different default from read_table 2552 encoding : string, optional 2553 a string representing the encoding to use if the contents are pandas/_libs/src/inference.pyx in pandas._libs.lib.map_infer() /usr/local/lib/python2.7/dist-packages/pysocialwatcher/utils.pyc in (x) 327 print_info("Computing Audience column") 328 collection_dataframe["audience"] = collection_dataframe["response"].apply( --> 329 lambda x: process_audience_from_response(x)) 330 return collection_dataframe 331 /usr/local/lib/python2.7/dist-packages/pysocialwatcher/utils.pyc in process_audience_from_response(literal_response) 319 320 def process_audience_from_response(literal_response): --> 321 audience = json.loads(literal_response)["data"]["users"] 322 return int(audience) 323 /usr/lib/python2.7/json/__init__.pyc in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw) 337 parse_int is None and parse_float is None and 338 parse_constant is None and object_pairs_hook is None and not kw): --> 339 return _default_decoder.decode(s) 340 if cls is None: 341 cls = JSONDecoder /usr/lib/python2.7/json/decoder.pyc in decode(self, s, _w) 362 363 """ --> 364 obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 365 end = _w(s, end).end() 366 if end != len(s): /usr/lib/python2.7/json/decoder.pyc in raw_decode(self, s, idx) 380 obj, end = self.scan_once(s, idx) 381 except StopIteration: --> 382 raise ValueError("No JSON object could be decoded") 383 return obj, end ValueError: No JSON object could be decoded — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub , or mute the thread .

Thanks for following up. The response generated seems to contain a lot of Javascript but no JSON. Sample attached. response.txt

Is this only in one request or this response is in all requests? This is a .html page with nothing related to the query(it's about some Facebook conference).

Matheus Araújo M.Sc. in Computer Science Pursuing Ph.D. at University of Minnesota.

On Mon, May 14, 2018 at 2:35 AM, Ian notifications@github.com wrote:

Thanks for following up. The response generated seems to contain a lot of Javascript but no JSON. Sample attached. response.txt https://github.com/maraujo/pySocialWatcher/files/1999740/response.txt

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/maraujo/pySocialWatcher/issues/9#issuecomment-388723874, or mute the thread https://github.com/notifications/unsubscribe-auth/AH2soWQXigS241hSfx7mfxh9rG48oE3xks5tyTO5gaJpZM4T8lOA .

This is from a single request that is stored in a temporary file during collection (from watcherAPI.run_data_collection:

import pandas as pd
TMP_FILE = 'dataframe_collecting_1526161260.csv'
TMP_DATA = pd.read_csv(TMP_FILE, index_col=0)
print(TMP_DATA.loc[:, 'response'].iloc[0])

I would delete this request in the file and load the temporary file: dataframe_collecting_1526161260.csv Use the function load_data_and_continue_collection(temp_file) so, you don't need to query everything again.

Matheus Araújo M.Sc. in Computer Science Pursuing Ph.D. at University of Minnesota.

On Mon, May 14, 2018 at 12:26 PM, Ian notifications@github.com wrote:

This is from a single request that is stored in a temporary file during collection (from watcherAPI.run_data_collection:

import pandas as pd TMP_FILE = 'dataframe_collecting_1526161260.csv' TMP_DATA = pd.read_csv(TMP_FILE, index_col=0) print(TMP_DATA.loc[:, 'response'].iloc[0])

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/maraujo/pySocialWatcher/issues/9#issuecomment-388897539, or mute the thread https://github.com/notifications/unsubscribe-auth/AH2soT2smtjfevT2hoSQApkjiOg7DfPGks5tyb5JgaJpZM4T8lOA .

Thanks for the tip. I'm starting to think that it's my access token that is malfunctioning. Do you have any basic queries that I could submit to the Graph Explorer API directly to see if my access token works?

Sorry, but I don't have queries for the Graph Explorer API.

Matheus Araújo M.Sc. in Computer Science Pursuing Ph.D. at University of Minnesota.

On Tue, May 15, 2018 at 10:21 AM, Ian notifications@github.com wrote:

Thanks for the tip. I'm starting to think that it's my access token that is malfunctioning. Do you have any basic queries that I could submit to the Graph Explorer API directly to see if my access token works?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/maraujo/pySocialWatcher/issues/9#issuecomment-389206737, or mute the thread https://github.com/notifications/unsubscribe-auth/AH2soTUJy-01uTn1t1jurpsuh8wP063yks5tyvJpgaJpZM4T8lOA .

Update: it turns out I was using my user ID for the account number, when I should have been using the account number for the Facebook Ads API. Very confusing! Thanks for your patience.

Thanks, @Ian for showing the solution.

Matheus Araújo M.Sc. in Computer Science Pursuing Ph.D. at University of Minnesota.

On Fri, May 18, 2018 at 10:55 AM, Ian notifications@github.com wrote:

Closed #9 https://github.com/maraujo/pySocialWatcher/issues/9.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/maraujo/pySocialWatcher/issues/9#event-1634605063, or mute the thread https://github.com/notifications/unsubscribe-auth/AH2soQ5cfTGZqhLLg3uXoHm1TenfQq3aks5tzu78gaJpZM4T8lOA .

maraujo / pySocialWatcher

Response doesn't return JSON #9