search

ibrigadir / opendatakit

Automatically exported from code.google.com/p/opendatakit
0 stars 0 forks source link

Migration problems from Aggregate 0.9.x to Aggregate 1.0 #406

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
On Mon, Oct 31, 2011 at 6:34 AM, LETSGO CeLeKT <letsgo.space@gmail.com> wrote:

    Thank you for the new Briefcase Beta App.
    We are now able to download the data from our two old App Engine
    Aggregates (v.0.9.8). We had lots of warnings and "will be ignored"
    messages in the logs, but all the data seems to be downloaded though.

    We followed Mitch's guide and edited every xml form and uploaded it
    manually to our new ODK Aggregate (v1.0 RC4, PostgreSQL).
    The data we downloaded is from two App Engine Aggregates (both
    v0.9.8).
    When we try to upload the data downloaded from the first Aggregate we
    receive following error messages in Briefcase for every form we
    upload, but all the data seem to be there though. Except for the
    "Starttime" and "Endtime". I checked and compared old and newer
    submission.xml files and the date format has changed completely, e.g.
    from "<StartTime>Tue May 25 13:55:30 UTC 2010</StartTime>" to
    "<StartTime>2011-10-05 10:46:38.192</StartTime>" (the examples are
    from different submission.xml files).

    Log:
    "Preparing for upload of form definition with 0 media attachments
    Uploading form definition with no media attachments
    Preparing for upload of submission (1 of 21) with 4 media attachments
    Uploading submission (1 of 21) and media files 1 through 3 of 4 media
    attachments
    UPLOAD FAILED: Submission upload failed.  Detailed error: Unexpected
    exception: Original submission file's submissionDate does not match
    that on server! C:\Users\...\submission.xml
    ...
    Uploading submission (21 of 21) and 1 remaining media attachments
    UPLOAD FAILED: Submission upload failed.  Detailed error: Unexpected
    exception: Original submission file's submissionDate does not match
    that on server! C:\Users\...\submission.xml
    Partially successful upload..."

    But when we try to upload the second server's data we only get a "Form
    does not exist" error for every form. Even though we see the forms on
    the Aggregate webpage, in the PostgreSQL database AND we can download
    and submit them with ODK Collect (1.1.7RC2).

    Unfortunately, we don't get a more detailed error message. So we
    really don't know what to do next.

    On Oct 21, 10:53 pm, Mitch Sundt <msu...@cs.washington.edu> wrote:
    > I've created an issue to track this:http://code.google.com/p/opendatakit/issues/detail?id=385
    >
    > Mitch
    >
    > On Wed, Oct 19, 2011 at 5:04 PM, Mitch Sundt <msu...@cs.washington.edu>wrote:
    >
    >
    >
    >
    >
    >
    >
    >
    >
    > > I'll look into this after I get RC3 out... .  This is an issue within the
    > > Briefcase App.
    >
    > > On Wed, Oct 19, 2011 at 3:03 AM, LETSGO CeLeKT <letsgo.sp...@gmail.com>wrote:
    >
    > >> We upgraded our Aggregate from 0.9.5 to 0.9.8 but we are still not
    > >> able to download our data with Briefcase.
    > >> Briefcase downloads the forms but all the submission data is missing!
    >
    > >> The error is still the same (of course):
    > >> "Fetching form definition
    > >> preparing to retrieve instance data
    > >> Error parsing form definition: xmlns attribute for the data model is
    > >> not well-formed: 'WaterQuality_openhouse' should be of the form
    > >> xmlns="http://your.domain.org/formId"
    > >> Consider defining the formId using the 'id' attribute instead of the
    > >> 'xmlns' attribute (id="formId")"
    >
    > >> So right now we are not able to follow the guide Mitch suggested ("you
    > >> should then be able to run Briefcase and choose to upload all data
    > >> from the local briefcase").
    >
    > >> Any ideas?
    >
    > >> On Oct 18, 2:37 pm, Yaw Anokwa <yano...@gmail.com> wrote:
    > >> > you can upgrade from .95 to .98 without loosing any data. it's only
    > >> > .9x to 1.0 that requires using briefcase.
    >
    > >> > On Tue, Oct 18, 2011 at 00:07, LETSGO CeLeKT <letsgo.sp...@gmail.com>
    > >> wrote:
    > >> > > Mitch, thank you very much for this detailed guide!
    >
    > >> > > We really don't want to lose our data.
    > >> > > Is it possible to install the 0.9.8 Aggregate over the 0.9.5 without
    > >> > > any changes to the database?
    >
    > >> > > On Oct 18, 1:39 am, "W. Brunette" <wbrune...@gmail.com> wrote:
    > >> > >> Also note u need to upgrade your Aggregate 0.9.5 to 0.9.8
    > >> > >> On Oct 17, 2011 8:22 AM, "LETSGO CeLeKT" <letsgo.sp...@gmail.com>
    > >> wrote:
    >
    > >> > >> > Hi,
    >
    > >> > >> > we try to transfer data from one ODK Aggregate to another and/or to
    > >> a
    > >> > >> > Briefcase directory on the filesystem, but we are not able to do
    > >> so.
    > >> > >> > The Source is an ODK Aggregate v0.9.5 on Google App Engine. Our new
    > >> > >> > server is a local Tomcat with ODK Aggregate v1.0 RC2 and a
    > >> PostgreSQL
    > >> > >> > database.
    >
    > >> > >> > We always receive this error message on every form we have:
    > >> > >> > "Fetching form definition
    > >> > >> > preparing to retrieve instance data
    > >> > >> > Error parsing form definition: xmlns attribute for the data model
    > >> is
    > >> > >> > not well-formed: 'Biodiversity' should be of the form
    > >> xmlns="http://
    > >> > >> > your.domain.org/formId"
    > >> > >> > Consider defining the formId using the 'id' attribute instead of
    > >> the
    > >> > >> > 'xmlns' attribute (id="formId")"
    >
    > >> > >> > How can we transfer/download data without changing every xml form?
    >
    > >> > >> > Regards!
    >
    > >> > >> > --
    > >> > >> > Post: opendatakit@googlegroups.com
    > >> > >> > Unsubscribe: opendatakit+unsubscribe@googlegroups.com
    > >> > >> > Options:http://groups.google.com/group/opendatakit?hl=en
    >
    > >> > > --
    > >> > > Post: opendatakit@googlegroups.com
    > >> > > Unsubscribe: opendatakit+unsubscribe@googlegroups.com
    > >> > > Options:http://groups.google.com/group/opendatakit?hl=en
    >
    > >> --
    > >> Post: opendatakit@googlegroups.com
    > >> Unsubscribe: opendatakit+unsubscribe@googlegroups.com
    > >> Options:http://groups.google.com/group/opendatakit?hl=en
    >
    > > --
    > > Mitch Sundt
    > > Software Engineer
    > >http://www.OpenDataKit.org
    > > University of Washington
    > > mitchellsu...@gmail.com
    >
    > --
    > Mitch Sundt
    > Software Engineerhttp://www.OpenDataKit.org
    > University of Washington
    > mitchellsu...@gmail.com

    --

Original issue reported on code.google.com by mitchellsundt@gmail.com on 31 Oct 2011 at 4:44

GoogleCodeExporter commented 9 years ago 
Can you double-check that the form XML file is named the same as the form 
directory on your local machine?  I.e., that you have:

forms\Geo Tagger v2\Geo Tagger v2.xml

The form definition filename must have exactly the same name portion as the 
enclosing folder.

Original comment by mitchellsundt@gmail.com on 31 Oct 2011 at 4:51

GoogleCodeExporter commented 9 years ago 
Hi,

I attached 3 files:
- submission_OLD.xml: a submission we downloaded with Briefcase from Aggregate 
v0.9.8.
- submission_NEW.xml: the same submission after uploaded with Briefcase to 
Aggregate v1.0RC4.
- submission_NEW_with_date.xml: a submission we uploaded using ODK Collect 
1.1.7RC2.

Hope this helps!

Original comment by letsgo.s...@gmail.com on 1 Nov 2011 at 10:15

Attachments:

GoogleCodeExporter commented 9 years ago 
OK.  This looks like a localization issue. Thank you very much for the 
submission files.  They help greatly.

Note that dates/times are NOT being successfully uploaded into your PostgreSQL 
instance.  The date is being lost (e.g., submission_NEW.xml has no values for 
<StartTime> and <EndTime> unlike submission_OLD.xml).

The short-term fix is likely to configure Tomcat to run in the US English 
domain. 
http://stackoverflow.com/questions/1153521/setting-default-locale-for-tomcat-ser
vice-in-windows-xp

Original comment by mitchellsundt@gmail.com on 1 Nov 2011 at 5:39

GoogleCodeExporter commented 9 years ago 
This requires an update to 0.9.8 (now 0.9.8.1), as well as updates to Aggregate 
1.0 (Aggregate 1.0 Production) and Briefcase (Briefcase 1.0 Beta 2).  Updates 
have been applied and I've verified that the dates are now in a non-localized 
standard format so that the migration should work as intended.

Original comment by mitchellsundt@gmail.com on 3 Nov 2011 at 12:53

GoogleCodeExporter commented 9 years ago 
Thank you very much for the updates!
We upgraded all of our servers and programs and now everything works fine!
So all the issues with date formats are gone and there was not a single error 
displayed during the Briefcase process.

Original comment by letsgo.s...@gmail.com on 3 Nov 2011 at 9:48

GoogleCodeExporter commented 9 years ago 
Wonderful!  Closing this issue.

Original comment by mitchellsundt@gmail.com on 3 Nov 2011 at 4:55