bonez56
commented
1 month ago I have exactly the same issue.
Closed maxime1992 closed 1 month ago
bonez56
commented
1 month ago I have exactly the same issue.
Freika
commented
1 month ago Without logs with errors from dawarich_sidekiq and a sample file you're trying to import, unfortunately it's impossible to debug this problem
LaustDengsoe
commented
1 month ago I am experiencing the same problem :)
The following is from the log of my sidekiq docker. I have redacted some of the data, but I think there should still be suffient data. If not please say so🤗 It looks like all the failed points fail with the same error.
2024-08-12T12:26:55.300Z pid=44 tid=2430k WARN: app/jobs/import_google_takeout_job.rb:8:inperform'
2024-08-12T12:26:55.301Z pid=44 tid=243bo class=ImportGoogleTakeoutJob jid=0e1ac9bf0dbf8485b77b64d4 INFO: start
D, [2024-08-12T12:26:55.309439 #44] DEBUG -- : Import Load (1.0ms) SELECT "imports".* FROM "imports" WHERE "imports"."id" = $1 LIMIT $2 [["id", 129], ["LIMIT", 1]]
I, [2024-08-12T12:26:55.312183 #44] INFO -- : Performing ImportGoogleTakeoutJob (Job ID: 9c97aaa3-5eaf-45bc-8609-21b159a57b6f) from Sidekiq(imports) enqueued at 2024-08-12T11:59:04.661433317Z with arguments: #<GlobalID:0x000014b3ace666e8 @uri=#<URI::GID gid://dawarich/Import/129>>, "{\"latitudeE7\":XXXXXXXXX,\"longitudeE7\":XXXXXXXXX,\"accuracy\":40,\"source\":\"WIFI\",\"deviceTag\":XXXXXXXXX,\"timestamp\":\"2014-01-13T21:28:00.548Z\"}"
E, [2024-08-12T12:26:55.313650 #44] ERROR -- : Error performing ImportGoogleTakeoutJob (Job ID: 9c97aaa3-5eaf-45bc-8609-21b159a57b6f) from Sidekiq(imports) in 6.74ms: ArgumentError (You are passing an instance of ActiveRecord::Base to find. Please pass the id of the object by calling .id.):
/var/app/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.4/lib/active_record/relation/finder_methods.rb:508:in find_one' /var/app/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.4/lib/active_record/relation/finder_methods.rb:499:infind_with_ids'
/var/app/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.4/lib/active_record/relation/finder_methods.rb:92:in find' /var/app/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.4/lib/active_record/querying.rb:23:infind'
/var/app/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.4/lib/active_record/core.rb:250:in find' /var/app/app/jobs/import_google_takeout_job.rb:8:inperform'
/var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/execution.rb:69:in block in _perform_job' /var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/callbacks.rb:121:inblock in run_callbacks'
/var/app/vendor/bundle/ruby/3.2.0/gems/i18n-1.14.5/lib/i18n.rb:351:in with_locale' /var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/translation.rb:9:inblock (2 levels) in instance_exec' /var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/callbacks.rb:130:inblock in run_callbacks'
/var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/core_ext/time/zones.rb:65:in use_zone' /var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/timezones.rb:9:inblock (2 levels) in instance_exec' /var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/callbacks.rb:130:inblock in run_callbacks'
/var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/callbacks.rb:141:in run_callbacks' /var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/execution.rb:68:in_perform_job'
/var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/instrumentation.rb:32:in _perform_job' /var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/execution.rb:52:inperform_now'
/var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/instrumentation.rb:26:in block in perform_now' /var/app/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.4/lib/active_record/railties/job_runtime.rb:13:inblock in instrument'
/var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/instrumentation.rb:40:in block in instrument' /var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/notifications.rb:206:inblock in instrument'
/var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/notifications/instrumenter.rb:58:in instrument' /var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/notifications.rb:206:ininstrument'
/var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/instrumentation.rb:39:in instrument' /var/app/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.3.4/lib/active_record/railties/job_runtime.rb:11:ininstrument'
/var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/instrumentation.rb:26:in perform_now' /var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/logging.rb:18:inblock in perform_now'
/var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/logging.rb:27:in tag_logger' /var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/logging.rb:18:inperform_now'
/var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/execution.rb:30:in block in execute' /var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/callbacks.rb:121:inblock in run_callbacks'
/var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/railtie.rb:67:in block (4 levels) in <class:Railtie>' /var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/reloader.rb:77:inblock in wrap'
/var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/execution_wrapper.rb:88:in wrap' /var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/reloader.rb:74:inwrap'
/var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/railtie.rb:66:in block (3 levels) in <class:Railtie>' /var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/callbacks.rb:130:ininstance_exec'
/var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/callbacks.rb:130:in block in run_callbacks' /var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/callbacks.rb:141:inrun_callbacks'
/var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/execution.rb:28:in execute' /var/app/vendor/bundle/ruby/3.2.0/gems/activejob-7.1.3.4/lib/active_job/queue_adapters/sidekiq_adapter.rb:70:inperform'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:210:in execute_job' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:180:inblock (4 levels) in process'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/middleware/chain.rb:180:in traverse' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/middleware/chain.rb:183:inblock in traverse'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/metrics/tracking.rb:26:in track' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/metrics/tracking.rb:126:incall'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/middleware/chain.rb:182:in traverse' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/middleware/chain.rb:173:ininvoke'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:179:in block (3 levels) in process' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:140:inblock (6 levels) in dispatch'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/job_retry.rb:113:in local' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:139:inblock (5 levels) in dispatch'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/rails.rb:16:in block in call' /var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/reloader.rb:77:inblock in wrap'
/var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/execution_wrapper.rb:92:in wrap' /var/app/vendor/bundle/ruby/3.2.0/gems/activesupport-7.1.3.4/lib/active_support/reloader.rb:74:inwrap'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/rails.rb:15:in call' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:135:inblock (4 levels) in dispatch'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:271:in stats' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:130:inblock (3 levels) in dispatch'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/job_logger.rb:13:in call' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:129:inblock (2 levels) in dispatch'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/job_retry.rb:80:in global' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:128:inblock in dispatch'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/job_logger.rb:39:in prepare' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:127:indispatch'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:178:in block (2 levels) in process' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:177:inhandle_interrupt'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:177:in block in process' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:176:inhandle_interrupt'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:176:in process' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:82:inprocess_one'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/processor.rb:72:in run' /var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/component.rb:10:inwatchdog'
/var/app/vendor/bundle/ruby/3.2.0/gems/sidekiq-7.2.4/lib/sidekiq/component.rb:19:in block in safe_thread' 2024-08-12T12:26:55.315Z pid=44 tid=2432s class=ImportGoogleTakeoutJob jid=efce86b1dcf6467dc5eff53d elapsed=0.209 INFO: fail 2024-08-12T12:26:55.315Z pid=44 tid=2432s WARN: {"context":"Job raised exception","job":{"retry":false,"queue":"imports","wrapped":"ImportGoogleTakeoutJob","args":[{"job_class":"ImportGoogleTakeoutJob","job_id":"9c97aaa3-5eaf-45bc-8609-21b159a57b6f","provider_job_id":null,"queue_name":"imports","priority":null,"arguments":[{"_aj_globalid":"gid://dawarich/Import/129"},"{\"latitudeE7\":XXXXXXXXX,\"longitudeE7\":XXXXXXXXX,\"accuracy\":40,\"source\":\"WIFI\",\"deviceTag\":XXXXXXXX,\"timestamp\":\"2014-01-13T21:28:00.548Z\"}"],"executions":0,"exception_executions":{},"locale":"en","timezone":"Europe/Berlin","enqueued_at":"2024-08-12T11:59:04.661433317Z","scheduled_at":null}],"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","jid":"efce86b1dcf6467dc5eff53d","created_at":1723463944.6614792,"enqueued_at":1723463944.6615067}} 2024-08-12T12:26:55.315Z pid=44 tid=2432s WARN: ArgumentError: You are passing an instance of ActiveRecord::Base tofind. Please pass the id of the object by calling.id.
bpmich
commented
1 month ago Looks to be the same issue as #152
Freika
commented
1 month ago @maxime1992 @LaustDengsoe @bonez56 @bpmich can you please send me the anonymized version of the file you're trying to import?
Freika
commented
1 month ago LaustDengsoe
commented
1 month ago It has been fixed for me! Would you still like a version of the file I'm trying to import?
Freika
commented
1 month ago @LaustDengsoe not anymore, since your problem was fixed :)
I'm closing this issue, let me know if there are still problems :)
Describe the bug Importing data from Google takeout and I get 100% "failed" in Sidekiq dashboard.
Version 0.9.9
To Reproduce I have launched the app with the docker compose for the first time and followed the manual import steps to import the Records.json file manually (not through the UI). I have tried to set
BACKGROUND_PROCESSING_CONCURRENCY: 1but same.Expected behavior No errors in Sidekiq
Screenshots
Logs
The logs seems to be fine in the app:
But the Sidekiq UI says otherwise?
Additional context None. I'm not too sure what are the errors in Sidekiq and how to fix those.
Thanks!