search

CivicSpleen / ambry

A comprehensive data package manager
BSD 2-Clause "Simplified" License
4 stars 5 forks source link

oshpd.ca.gov-facilities-cross.yaml import error: ResourceInvalidError: Resource is invalid: /chhs.data.ca.gov/api/views/tthg-z4mf #114

Closed nmb10 closed 8 years ago

nmb10 commented 8 years ago

run:

python load_pre10.py ../../pre-10-bundles/converted/oshpd.ca.gov-facilities-cross/oshpd.ca.gov-facilities-cross.yaml

error:

Starting import ../../pre-10-bundles/converted/oshpd.ca.gov-facilities-cross/oshpd.ca.gov-facilities-cross.yaml...
Loading bundle: oshpd.ca.gov-facilities-cross-0.0.4~d03U004
INFO oshpd.ca.gov-facilities-cross ---- Synchronized ----
Starting ingest...
INFO oshpd.ca.gov-facilities-cross Ingesting: facility_cross from https://chhs.data.ca.gov/api/views/tthg-z4mf/rows.csv
INFO oshpd.ca.gov-facilities-cross Ingested: facility_cross.mpr
INFO oshpd.ca.gov-facilities-cross Ingesting: meta from https://chhs.data.ca.gov/api/views/tthg-z4mf
Traceback (most recent call last):
  File "load_pre10.py", line 251, in <module>
    main()
  File "load_pre10.py", line 220, in main
    _ingest(b)
  File "load_pre10.py", line 177, in _ingest
    b.ingest(force=force, clean_files=clean_files)
  File "/home/nmb10/projects/ambry_project/ambry/bundle/bundle.py", line 873, in ingest
    clean=force, account_accessor=account_accessor)
  File "/home/nmb10/.virtualenvs/ambry/local/lib/python2.7/site-packages/ambry_sources/download.py", line 42, in get_source
    cache_path, download_time = download(spec.url, cache_fs, account_accessor, clean=clean)
  File "/home/nmb10/.virtualenvs/ambry/local/lib/python2.7/site-packages/ambry_sources/download.py", line 170, in download
    cache_fs.remove(cache_path)
  File "/home/nmb10/.virtualenvs/ambry/local/lib/python2.7/site-packages/fs/errors.py", line 257, in wrapper
    return func(self,*args,**kwds)
  File "/home/nmb10/.virtualenvs/ambry/local/lib/python2.7/site-packages/fs/osfs/__init__.py", line 275, in remove
    os.remove(sys_path)
fs.errors.ResourceInvalidError: Resource is invalid: /chhs.data.ca.gov/api/views/tthg-z4mf
nmb10 commented 8 years ago

What should I do to fix that? The error occurs for that source:

    sources:
    ...
        meta:
            filetype: socrata/json
            is_loadable: false
            url: https://chhs.data.ca.gov/api/views/tthg-z4mf
ericbusboom commented 8 years ago

If 'is_loadable' is 'false' set 'reftype' to 'ref'

nmb10 commented 8 years ago

Fixed in to_v1.py script of pre-10-bundles repo.