akvo / akvo-lumen

Make sense of your data
https://akvo.org/akvo-lumen
GNU Affero General Public License v3.0
63 stars 18 forks source link

Null cause from instance - error when importing a dataset #3096

Closed janagombitova closed 2 years ago

janagombitova commented 3 years ago

When trying to import a specific Flow form to Lumen I get the following error.

Screenshot 2021-01-28 at 08 21 43 Lumen: a4a.akvolumen.org Flow: aquaforall.akvoflow.org Dataset: HWTS & utilities pilot Ethiopia->Ethiopia Sales Tracker The support@akvolumen.org user has access to both instances

I tried importing other datasets from the Flow instance to the same Lumen instance and that worked without issues. The dataset at first glance looks fine.

tangrammer commented 3 years ago

specific form submissions have date values in an incorrect/unexpected akvo-flow-api format

we can reproduce the problem easily

(let [date-format-2 (SimpleDateFormat. "dd-MM-yyyy")]
  (.parse date-format-2 "26/12/2016"))
1. Unhandled java.text.ParseException
   Unparseable date: "26/12/2016"

           DateFormat.java:  395  java.text.DateFormat/parse
                      REPL:  113  akvo.lumen.lib.import.flow-v3/eval61726
                      REPL:  112  akvo.lumen.lib.import.flow-v3/eval61726
             Compiler.java: 7177  clojure.lang.Compiler/eval
             Compiler.java: 7132  clojure.lang.Compiler/eval
                  core.clj: 3214  clojure.core/eval
                  core.clj: 3210  clojure.core/eval
    interruptible_eval.clj:   91  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  437  clojure.main/repl/read-eval-print/fn
                  main.clj:  437  clojure.main/repl/read-eval-print
                  main.clj:  458  clojure.main/repl/fn
                  main.clj:  458  clojure.main/repl
                  main.clj:  368  clojure.main/repl
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  155  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  190  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  189  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  834  java.lang.Thread/run

Using expected date format works fine:

(let [date-format-2 (SimpleDateFormat. "dd-MM-yyyy")]
  (.parse date-format-2 "26-12-2016"))

#inst "2016-12-26T00:00:00.000-00:00"

Sentry is capturing the problem too: https://sentry.io/organizations/akvo-foundation/issues/2183927016/?project=157366&referrer=alert_email

janagombitova commented 3 years ago

Thanks @tangrammer

Looking at the dataset, column L Date of sale all submissions made with form version2 have an odd value: Screenshot 2021-01-28 at 10 29 20

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.