CivicSpleen / ambry

A comprehensive data package manager
BSD 2-Clause "Simplified" License
4 stars 5 forks source link

sandiego.gov-opendsd-index.yaml import error: Failed to determine file type for source '2003_completed_permits'; unknown type 'xml' #110

Closed nmb10 closed 8 years ago

nmb10 commented 8 years ago

run:

python load_pre10.py ../../pre-10-bundles/converted/sandiego.gov-opendsd-index/sandiego.gov-opendsd-index.yaml

error:

Starting import ../../pre-10-bundles/converted/sandiego.gov-opendsd-index/sandiego.gov-opendsd-index.yaml...
Loading bundle: sandiego.gov-opendsd-index-0.0.3~d03f003
INFO sandiego.gov-opendsd-index ---- Synchronized ----
Starting ingest...
INFO sandiego.gov-opendsd-index Ingesting: 2003_completed_permits from http://ds.civicknowledge.org.s3.amazonaws.com/sandiego.gov/opendsd/2003/Annual_2003_DSD_Permits_Completed.xml
Traceback (most recent call last):
  File "load_pre10.py", line 251, in <module>
    main()
  File "load_pre10.py", line 220, in main
    _ingest(b)
  File "load_pre10.py", line 177, in _ingest
    b.ingest(force=force, clean_files=clean_files)
  File "/home/nmb10/projects/ambry_project/ambry/bundle/bundle.py", line 873, in ingest
    clean=force, account_accessor=account_accessor)
  File "/home/nmb10/.virtualenvs/ambry/local/lib/python2.7/site-packages/ambry_sources/download.py", line 76, in get_source
    .format(spec.name, file_type))
ambry_sources.sources.exceptions.SourceError: Failed to determine file type for source '2003_completed_permits'; unknown type 'xml'
ericbusboom commented 8 years ago

This source is XML, which isn't downloadable without special processing in the bundle, so make it a non downloadable reference type.