Related bulk upload 116

[TeriForey]

Description of Change

Using existing bulk upload steps, added processes to extract related resource information from a sheet called 'RELATIONS'. Can take in row number (if it also contains new resources that are being uploaded), EAMENA_IDs or the resource uuid. Excel file will be parsed into a .relations file (along with a .arches file) and uploaded using the load_resources management command. It's also possible to upload only relations without any resources (new option in resource type dropdown list), in which case it runs the load_relations management command.

Issues Solved

116

Types of changes

• [ ] Bugfix (non-breaking change which fixes an issue)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

• [ ] Unit tests pass locally with my changes
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] I have added necessary documentation (if appropriate)

Further comments

Might want to pull out the sheet name into the settings file. Might also want to speed up the process when uploading only relations and once validated just import them directly into the database, instead of creating a file and then running a management command. Example excel files will be shared in the Slack channel.

[mradamcox]

Just tested this out locally, and it works great. I think it's reasonable to just leave RELATIONS as a convention, as there are already a few conventions involved in the Excel file upload. As to bypassing the .relations file and loading directly, in my opinion it's nice to have the .relations file exist as a record, for interoperability with other Arches installations... not that we really expect to be loading this stuff into a deployment without the bulk upload feature, but having the .relations files does make it possible.

I'm going to merge this in now, and then cherry-pick these commits to a new branch so we can get them deployed to Al Ula. Thanks for this work @TeriForey!

[mradamcox]

@pascalflohr this new feature has been deployed to Al Ula, you can now upload relationships using a new RELATIONS sheet in an excel upload file.

There are two methods for this:

1) Only uploading relations. Your excel file should looks like the following (in a RELATIONS sheet) with no other sheets. Note that EAMENA_IDs are used here, but in Al Ula you would use IDIHA ids. You can also use the resource UUIDs directly from Arches.

RESOURCEID_FROM	RESOURCEID_TO	START_DATE	END_DATE	RELATION_TYPE
EAMENA-0000003	EAMENA-0000001|EAMENA-0000004	2018-04-28|2018-03-27	2018-04-29|2018-05-04	has the same spatial unit as|has the same spatial unit as
EAMENA-0000009	EAMENA-0000004|EAMENA-0000005|EAMENA-0000006|EAMENA-0000007	x|x|x|x	x|x|x|x	is adjacent to|has the same spatial unit as|is intersecting|is intersecting

2) Uploading relationships along with resources in an excel file. In that case your RELATIONS sheet should look like the following. Note that there is no FROM column, as this FROM is the resource represented by the same row in all the other sheets. Also, note the number 3 in the first resource refers to row 3 of the spreadsheet, i.e. the second resource in the list (as the headers are row 1).

RESOURCEID_TO	START_DATE	END_DATE	RELATION_TYPE	NOTES
3	2018-04-28	2018-04-29	has the same spatial unit as	Got a note
EAMENA-0000001	x	x	has the same spatial unit as	None

---

@TeriForey can help if you have any trouble with this, and can hopefully pitch in if I've incorrectly summarized anything here.

[pascalflohr]

That is great @mradamcox, very useful functionality

[pascalflohr]

@TeriForey I am unable to get past the validation stage, the wheel keeps turning and nothing appears to happen. I tried with IDIHA-numbers (equivalent to EAMENA) and Arches UUIDs, as in the attached files. What am I doing wrong? B3_relationships_test_UUID.xlsx B3_relationships_test.xlsx

[TeriForey]

@pascalflohr rename the sheet to RELATIONS and it should pick it up - you'll also want to make sure you've selected the 'Resource Relationships' option in the dropdown box (step 2 - Choose the Configuration).

[pascalflohr]

@TeriForey thank you, that was the problem indeed. It works, brilliant!