search

openownership / cove-bods

Check that your data complies with the Beneficial Ownership Data Standard (BODS) using our open source data review tool
https://datareview.openownership.org/
Other
4 stars 2 forks source link

Add information to home page about file size limits #80

Closed odscjames closed 3 years ago

odscjames commented 3 years ago

From https://github.com/openownership/cove-bods/issues/69#issuecomment-964223981

Update documentation on datareview.openownership.org to recommend current use of DRT only for .json files smaller than 100mb. Prompt users to email support@openownership.org if they want help to analyse larger files. Exact language TBC.

Assuming this is new content in the "Using the data review tool " section, this shouldn't take long to sort out - as long as reviewing and approving the new content takes, essentially.

Would we need to provide different guidance for maximum recommended file sizes across the four accepted formats?

In theory yes, but we could start with a low limit that would apply to all.

StephenAbbott commented 3 years ago

Should I just submit a PR with proposed changes in base.html and then assign to you, @odscjames?

I plucked the 100mb limit at random assuming that this might take around 6 minutes to process if the 316.2MB BODS JSON file from Latvia took 18 minutes. Any tips on what we think a suitable low limit would be?

odscjames commented 3 years ago

Should I just submit a PR with proposed changes in base.html and then assign to you, @odscjames?

Yes, thanks.

Any tips on what we think a suitable low limit would be?

6 minutes is a long time for users to wait with no UI feedback so does this need to be lower?

In some ways I'd suggest the exact number is less important and what is more important is the terms of the rough scale it communicates to people that this tool is suitable for. (If the number is X and someone has a file that is X+a-little-bit I'd expect them to just try it anyway, I would :smile: )

10MB maybe, at the moment? That should cover any sample or small data sets, but sets the expectation that for larger data sets this may not work - while the message provides a clear path for them to follow instead (hopefully).

Easy to change the message later.

StephenAbbott commented 3 years ago

@odscjames PR now submitted https://github.com/openownership/cove-bods/pull/82

odscjames commented 3 years ago

Now live