search

solectrus / csv-importer

CLI tool to upload historical photovoltaics data to InfluxDB
MIT License
6 stars 5 forks source link

bundler: failed to load command: src/main.rb (src/main.rb) #2

Closed spirit0607 closed 2 years ago

spirit0607 commented 2 years ago

Hallo,

vielen Dank für die tolle App! Die Installation auf einer Synology Diskstation hat wunderbar funktioniert. Leider bekomme ich den Import der alten Daten nicht hin. Ich bekomme die Fehlermeldung "bundler: failed to load command: src/main.rb (src/main.rb)"

Woran könnte das liegen?

root@DiskStation:/volume1/docker/history# docker run -it --rm            --env-file .env            -v /volume1/docker/solectrus/history/data:/data            ghcr.io/solectrus/senec-importer
SENEC CSV importer for SOLECTRUS
https://github.com/solectrus/senec-importer
Copyright (c) 2020,2022 Georg Ledermann, released under the MIT License

Using Ruby 3.1.2 on platform x86_64-linux-musl
Pushing to InfluxDB at http://localhost:8086, bucket my-solectrus-bucket

Importing data from /data ...
Importing /data/S53084641446072328540227639-week-25-2022.csv... bundler: failed to load command: src/main.rb (src/main.rb)
/senec-importer/src/import.rb:46:in `parse_kw': undefined method `sub' for nil:NilClass (NoMethodError)

    (row[string].sub(',', '.').to_f * 1_000).round
                ^^^^
    from /senec-importer/src/import.rb:74:in `record'
    from /senec-importer/src/import.rb:31:in `block in process'
    from /usr/local/lib/ruby/3.1.0/csv/table.rb:539:in `each'
    from /usr/local/lib/ruby/3.1.0/csv/table.rb:539:in `each'
    from /senec-importer/src/import.rb:28:in `map'
    from /senec-importer/src/import.rb:28:in `process'
    from /senec-importer/src/import.rb:11:in `block in run'
    from /senec-importer/src/import.rb:10:in `each'
    from /senec-importer/src/import.rb:10:in `run'
    from src/main.rb:20:in `<top (required)>'
    from /usr/local/bundle/gems/bundler-2.3.11/lib/bundler/cli/exec.rb:58:in `load'
    from /usr/local/bundle/gems/bundler-2.3.11/lib/bundler/cli/exec.rb:58:in `kernel_load'
    from /usr/local/bundle/gems/bundler-2.3.11/lib/bundler/cli/exec.rb:23:in `run'
    from /usr/local/bundle/gems/bundler-2.3.11/lib/bundler/cli.rb:483:in `exec'
    from /usr/local/bundle/gems/bundler-2.3.11/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
    from /usr/local/bundle/gems/bundler-2.3.11/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
    from /usr/local/bundle/gems/bundler-2.3.11/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
    from /usr/local/bundle/gems/bundler-2.3.11/lib/bundler/cli.rb:31:in `dispatch'
    from /usr/local/bundle/gems/bundler-2.3.11/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
    from /usr/local/bundle/gems/bundler-2.3.11/lib/bundler/cli.rb:25:in `start'
    from /usr/local/bundle/gems/bundler-2.3.11/exe/bundle:48:in `block in <top (required)>'
    from /usr/local/bundle/gems/bundler-2.3.11/lib/bundler/friendly_errors.rb:103:in `with_friendly_errors'
    from /usr/local/bundle/gems/bundler-2.3.11/exe/bundle:36:in `<top (required)>'
    from /usr/local/bundle/bin/bundle:25:in `load'
    from /usr/local/bundle/bin/bundle:25:in `<main>'
ledermann commented 2 years ago

@spirit0607 Willkommen im Club :-)

Wie ich gerade sehe, hat SENEC offenbar das Format der CSV-Dateien geringfügig geändert, die Spaltenbezeichnung lauten nun etwas anders, sodass es zu dem von Dir beschriebenen Fehler kommt.

Ich habe den Code gerade angepasst, mit dem aktuellen Docker-Image sollte es nun klappt:

$ docker pull ghcr.io/solectrus/senec-importer
$ docker run ....
spirit0607 commented 2 years ago

Hey vielen lieben Dank für die schnelle Antwort und die Hilfe. Das Update hat scheinbar funktioniert, allerdings bekomme ich nun folgende Fehlermeldung:

root@DiskStation:/volume1/docker/solectrus/history# docker run -it --rm            --env-file .env            -v /volume1/docker/solectrus/history/data:/data            ghcr.io/solectrus/senec-importer
SENEC CSV importer for SOLECTRUS
https://github.com/solectrus/senec-importer
Copyright (c) 2020,2022 Georg Ledermann, released under the MIT License

Using Ruby 3.1.2 on platform x86_64-linux-musl
Pushing to InfluxDB at http://192.168.178.26:8086, bucket my-solectrus-bucket

Importing data from /data ...
Importing /data/week-26-2022.csv... bundler: failed to load command: src/main.rb (src/main.rb)
/usr/local/bundle/gems/influxdb-client-2.7.0/lib/influxdb2/client/default_api.rb:106:in `_request': <html>\r (InfluxDB2::InfluxError)
<head><title>400 Request Header Or Cookie Too Large</title></head>\r
<body>\r
<center><h1>400 Bad Request</h1></center>\r
<center>Request Header Or Cookie Too Large</center>\r
<hr><center>nginx</center>\r
</body>\r
</html>\r
    from /usr/local/bundle/gems/influxdb-client-2.7.0/lib/influxdb2/client/default_api.rb:59:in `_post_text'
    from /usr/local/bundle/gems/influxdb-client-2.7.0/lib/influxdb2/client/write_api.rb:214:in `write_raw'
    from /usr/local/bundle/gems/influxdb-client-2.7.0/lib/influxdb2/client/write_api.rb:186:in `write'
    from /senec-importer/src/flux_writer.rb:17:in `push'
    from /senec-importer/src/flux_writer.rb:11:in `push'
    from /senec-importer/src/import.rb:36:in `process'
    from /senec-importer/src/import.rb:11:in `block in run'
    from /senec-importer/src/import.rb:10:in `each'
    from /senec-importer/src/import.rb:10:in `run'
    from src/main.rb:20:in `<top (required)>'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli/exec.rb:58:in `load'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli/exec.rb:58:in `kernel_load'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli/exec.rb:23:in `run'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli.rb:486:in `exec'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli.rb:31:in `dispatch'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli.rb:25:in `start'
    from /usr/local/bundle/gems/bundler-2.3.22/exe/bundle:48:in `block in <top (required)>'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/friendly_errors.rb:120:in `with_friendly_errors'
    from /usr/local/bundle/gems/bundler-2.3.22/exe/bundle:36:in `<top (required)>'
    from /usr/local/bundle/bin/bundle:25:in `load'
    from /usr/local/bundle/bin/bundle:25:in `<main>'
spirit0607 commented 2 years ago

Achso sorry, bevor ich die o.g. Fehlermeldung erhalten habe, habe ich versucht einen Reverse Proxy auf Port 8086 einzurichten, weil ich diese Fehlermeldung erhalten habe:

root@DiskStation:/volume1/docker/solectrus/history# docker run -it --rm            --env-file .env            -v /volume1/docker/solectrus/history/data:/data            ghcr.io/solectrus/senec-importer
SENEC CSV importer for SOLECTRUS
https://github.com/solectrus/senec-importer
Copyright (c) 2020,2022 Georg Ledermann, released under the MIT License

Using Ruby 3.1.2 on platform x86_64-linux-musl
Pushing to InfluxDB at http://localhost:8086, bucket my-solectrus-bucket

Importing data from /data ...
Importing /data/week-26-2022.csv... bundler: failed to load command: src/main.rb (src/main.rb)
/usr/local/lib/ruby/3.1.0/socket.rb:1214:in `__connect_nonblock': Failed to open TCP connection to localhost:8086 (Address not available - connect(2) for [::1]:8086) (Errno::EADDRNOTAVAIL)
    from /usr/local/lib/ruby/3.1.0/socket.rb:1214:in `connect_nonblock'
    from /usr/local/lib/ruby/3.1.0/socket.rb:56:in `connect_internal'
    from /usr/local/lib/ruby/3.1.0/socket.rb:137:in `connect'
    from /usr/local/lib/ruby/3.1.0/socket.rb:642:in `block in tcp'
    from /usr/local/lib/ruby/3.1.0/socket.rb:227:in `each'
    from /usr/local/lib/ruby/3.1.0/socket.rb:227:in `foreach'
    from /usr/local/lib/ruby/3.1.0/socket.rb:632:in `tcp'
    from /usr/local/lib/ruby/3.1.0/net/http.rb:998:in `connect'
    from /usr/local/lib/ruby/3.1.0/net/http.rb:976:in `do_start'
    from /usr/local/lib/ruby/3.1.0/net/http.rb:965:in `start'
    from /usr/local/lib/ruby/3.1.0/net/http.rb:1530:in `request'
    from /usr/local/bundle/gems/influxdb-client-2.7.0/lib/influxdb2/client/default_api.rb:82:in `_request'
    from /usr/local/bundle/gems/influxdb-client-2.7.0/lib/influxdb2/client/default_api.rb:59:in `_post_text'
    from /usr/local/bundle/gems/influxdb-client-2.7.0/lib/influxdb2/client/write_api.rb:214:in `write_raw'
    from /usr/local/bundle/gems/influxdb-client-2.7.0/lib/influxdb2/client/write_api.rb:186:in `write'
    from /senec-importer/src/flux_writer.rb:17:in `push'
    from /senec-importer/src/flux_writer.rb:11:in `push'
    from /senec-importer/src/import.rb:36:in `process'
    from /senec-importer/src/import.rb:11:in `block in run'
    from /senec-importer/src/import.rb:10:in `each'
    from /senec-importer/src/import.rb:10:in `run'
    from src/main.rb:20:in `<top (required)>'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli/exec.rb:58:in `load'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli/exec.rb:58:in `kernel_load'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli/exec.rb:23:in `run'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli.rb:486:in `exec'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli.rb:31:in `dispatch'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli.rb:25:in `start'
    from /usr/local/bundle/gems/bundler-2.3.22/exe/bundle:48:in `block in <top (required)>'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/friendly_errors.rb:120:in `with_friendly_errors'
    from /usr/local/bundle/gems/bundler-2.3.22/exe/bundle:36:in `<top (required)>'
    from /usr/local/bundle/bin/bundle:25:in `load'
    from /usr/local/bundle/bin/bundle:25:in `<main>'
/usr/local/lib/ruby/3.1.0/socket.rb:1214:in `__connect_nonblock': Address not available - connect(2) for [::1]:8086 (Errno::EADDRNOTAVAIL)
    from /usr/local/lib/ruby/3.1.0/socket.rb:1214:in `connect_nonblock'
    from /usr/local/lib/ruby/3.1.0/socket.rb:56:in `connect_internal'
    from /usr/local/lib/ruby/3.1.0/socket.rb:137:in `connect'
    from /usr/local/lib/ruby/3.1.0/socket.rb:642:in `block in tcp'
    from /usr/local/lib/ruby/3.1.0/socket.rb:227:in `each'
    from /usr/local/lib/ruby/3.1.0/socket.rb:227:in `foreach'
    from /usr/local/lib/ruby/3.1.0/socket.rb:632:in `tcp'
    from /usr/local/lib/ruby/3.1.0/net/http.rb:998:in `connect'
    from /usr/local/lib/ruby/3.1.0/net/http.rb:976:in `do_start'
    from /usr/local/lib/ruby/3.1.0/net/http.rb:965:in `start'
    from /usr/local/lib/ruby/3.1.0/net/http.rb:1530:in `request'
    from /usr/local/bundle/gems/influxdb-client-2.7.0/lib/influxdb2/client/default_api.rb:82:in `_request'
    from /usr/local/bundle/gems/influxdb-client-2.7.0/lib/influxdb2/client/default_api.rb:59:in `_post_text'
    from /usr/local/bundle/gems/influxdb-client-2.7.0/lib/influxdb2/client/write_api.rb:214:in `write_raw'
    from /usr/local/bundle/gems/influxdb-client-2.7.0/lib/influxdb2/client/write_api.rb:186:in `write'
    from /senec-importer/src/flux_writer.rb:17:in `push'
    from /senec-importer/src/flux_writer.rb:11:in `push'
    from /senec-importer/src/import.rb:36:in `process'
    from /senec-importer/src/import.rb:11:in `block in run'
    from /senec-importer/src/import.rb:10:in `each'
    from /senec-importer/src/import.rb:10:in `run'
    from src/main.rb:20:in `<top (required)>'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli/exec.rb:58:in `load'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli/exec.rb:58:in `kernel_load'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli/exec.rb:23:in `run'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli.rb:486:in `exec'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli.rb:31:in `dispatch'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli.rb:25:in `start'
    from /usr/local/bundle/gems/bundler-2.3.22/exe/bundle:48:in `block in <top (required)>'
    from /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/friendly_errors.rb:120:in `with_friendly_errors'
    from /usr/local/bundle/gems/bundler-2.3.22/exe/bundle:36:in `<top (required)>'
    from /usr/local/bundle/bin/bundle:25:in `load'
    from /usr/local/bundle/bin/bundle:25:in `<main>'
ledermann commented 2 years ago

Ok, es ist etwas tricky. Ich hatte übersehen, dass Du die Inhouse-Installation gemacht hattest, bei der sämtliche Komponenten auf der gleichen Maschine laufen. Die hierfür vorbereitete docker-compose.yml konfiguriert InfluxDB so, dass die Datenbank nicht von außen erreichbar ist.

Zur Lösung sind damit zwei Änderungen nötig:

1) In der docker-compose.yml muss der Port nach außen geöffnet werden, das geht mittels folgender Änderung:

influxdb:
  # .... restliche Konfiguration
  ports:
    - 8086:8086

Neustarten nicht vergessen: docker-compose up -d

2) Der Aufruf für den Import muss außerdem den richtigen Host verwenden - am einfachsten die IP-Adresse, über die die NAS im lokalen Netz erreichbar ist. Wenn die IP-Adresse z.B. 192.168.178.26 lautet, dann gelingt der Import wie folgt:

docker run -it --rm \
           --env-file .env \
           --env INFLUX_HOST=192.168.178.26 \
           -v /folder/with/csv-files:/data \
           ghcr.io/solectrus/senec-importer
spirit0607 commented 2 years ago

Super klasse! Jetzt hat es geklappt. Vielen lieben Dank für die tolle Unterstützung und für das ganze Projekt! :-)