search

getodk / central-backend

Node.js based backend for ODK Central
https://docs.getodk.org/central-intro/
Apache License 2.0
50 stars 75 forks source link

Missing rows in joined tables when exporting all submissions #271

Closed rmylb closed 4 years ago

rmylb commented 4 years ago

Hi, I'm facing out a problem, when I try to export all submissions in csv from frontend button or API. We use a repeat process in the form and only few rows are present in the joined tables (40, 45 rows instead 150 rows for example). Values are present in the database, but I can retrieve it in the csv export.
All entries seems to be present in the parent csv (16 submissions, as showed in the frontend table).

It's a fresh install of the lastest ODK-Central release. 

versions:
e6091155f39f6c27193747f15fc93a421793bb55
 e20603ce1548472981f5a11d471e3362eec1bd32 client (v0.8.0)
 f6502a04fa3f0e167df1b9c0d04b4775ecd75baa server (v0.8.1)

No error in docker-compose logs service : 

"GET /v1/projects/4/forms/tk3/submissions.csv.zip HTTP/1.0" 200 -

With ODK Briefcase, we can retrieve all data without problems.

Thanks for your help, tips and advices.

lognaturel commented 4 years ago

Sorry you’re experiencing this. Could you please share the form definition? If you run the export several times, do you always get the same row subset? Is this a form that has had multiple versions over time?

rmylb commented 4 years ago

You can find the form here (without media) : kollect_suivi_inv_taxon_3.txt

If I run the export several times, I get the same row subset. But if there is a new submission, I retrieve the new data but I loose the oldest, and the number of rows stay constant. True, this form has multiple versions.

Thanks, I appreciate your help.

lognaturel commented 4 years ago

Thanks for sharing the form definition. We have this on our radar and are going to try to get to it as soon as we can.

issa-tseng commented 4 years ago

hey @rylebrun, it looks like there are several csv attachments this form requires. i need to somehow get some data to try to recreate the problem you had; if i could either get some of your final (complete) submission data or the attachment files it would make solving this issue go quite a bit quicker. thanks!

lognaturel commented 4 years ago

Like @issa-tseng says, without the media for this form, it's hard to verify.

I think a forum post from Odil likely points to the same issue. I've reproduced on the test server with that form definition (credentials needed).

matthew-white commented 4 years ago

This should be resolved by #291. We've released this fix as part of v1.0.3. @rylebrun, let us know if you continue to experience an issue!