"Error: No schema for entity." while loading from opensanctions

[thimios]

I am getting an "Error: No schema for entity" while trying to load eu_fsf and us_ofac_sdn from opensanctions into an aleph instance.

This is the versions of python and the relevant packages I am using:

# python3 --version
Python 3.8.10

# pip show followthemoney
Name: followthemoney
Version: 2.7.3

# pip show alephclient
Name: alephclient
Version: 2.3.5

Here is what I am calling and the output I am getting:

# curl -s https://data.opensanctions.org/datasets/latest/eu_fsf/entities.ftm.json | /usr/local/bin/alephclient write-entities -f eu_fsf;
INFO:alephclient.cli:[eu_fsf] Bulk load entities: 1000...
Error: No schema for entity.

Also:

# curl -s https://data.opensanctions.org/datasets/latest/us_ofac_sdn/entities.ftm.json | /usr/local/bin/alephclient write-entities -f us_ofac_sdn;
INFO:alephclient.cli:[us_ofac_sdn] Bulk load entities: 1000...
INFO:alephclient.cli:[us_ofac_sdn] Bulk load entities: 2000...
INFO:alephclient.cli:[us_ofac_sdn] Bulk load entities: 3000...
INFO:alephclient.cli:[us_ofac_sdn] Bulk load entities: 4000...
INFO:alephclient.cli:[us_ofac_sdn] Bulk load entities: 5000...
Error: No schema for entity.

It seems that it comes from here: https://github.com/alephdata/followthemoney/blob/master/followthemoney/proxy.py#L61

[thimios]

@pudo any ideas what could be the issue here?

[pudo]

Usually a sign that the file you're trying to import is not a stream of FtM entities. Can you maybe debug this to the point where you see if there's non-ftm stuff in that file? Seems really weird.

[pudo]

I'm getting this:

(wrangle) ➜  ~ curl -s https://data.opensanctions.org/datasets/latest/us_ofac_sdn/entities.ftm.json | jq -r ".schema" | sort | uniq -c
7454 Address
 277 Airplane
1213 Company
 123 CryptoWallet
  95 Directorship
   7 Family
 403 Identification
 215 LegalEntity
  57 Membership
2482 Organization
1970 Ownership
1578 Passport
4726 Person
 816 Representation
9130 Sanction
 479 UnknownLink
 419 Vessel

[pudo]

Ooooh what version of FtM is your aleph instance using? Identification and CryptoWallet are pretty new. You may want to upgrade aleph or grep out those entities.

[Rosencrantz]

@pudo Thanks for jumping on this for us. @thimios Aleph is a couple of versions behind on it's FtM but this will be resolved in the next 3.11.x release of Aleph.

[thimios]

Upgrading Aleph to 3.11.0 resolved the issue!

[Rosencrantz]

Thanks for updating us Thimios, much appreciated