Closed case-k-git closed 3 years ago
this issue has been solved by increasing storage size.
Error: org.jruby.exceptions.RaiseException: (Errno::ENOSPC) /tmp/embulk_output_bigquery_20210726-7-1sosuuw.7.2004.jsonl
when I check rest of the storage capacity. there are not enough capacity
/dev/nvme0n1p1 485G 478G 7.1G 99% /
after increased the storage capacity the issue has been solved. there are not error messages of double release
/dev/nvme0n1p1 2.0T 632G 1.4T 32% /
2021-07-26 12:54:02.853 +0900 [INFO] (0108@[0:datatransfer]+dailybatch_bigquery_transfer+transfer_to_bigquery^sub+for-0=DATABASE=7=<database>+transfer_to_bigquery+transfer_target_tables^sub+for-0=TABLE=0=<table>+transfer_to_bigquery_private) io.digdag.core.agent.OperatorManager: sh>: embulk run -b /var/lib/embulk tasks/embulk/embulk_bigquery_load.yml.liquid
2021-07-26 12:54:02.861 +0900 [INFO] (0108@[0:datatransfer]+dailybatch_bigquery_transfer+transfer_to_bigquery^sub+for-0=DATABASE=7=<database>+transfer_to_bigquery+transfer_target_tables^sub+for-0=TABLE=0=<table>+transfer_to_bigquery_private) io.digdag.standards.command.DockerCommandExecutor: Pulling docker image <aws-account-id>.dkr.ecr.ap-northeast-1.amazonaws.com/<repo>/embulk
2021-07-26 03:54:18.638 +0000: Embulk v0.9.5
2021-07-26 03:54:24.744 +0000 [INFO] (main): Started Embulk v0.9.5
2021-07-26 03:54:25.266 +0000 [INFO] (0001:transaction): BUNDLE_GEMFILE is being set: "/var/lib/embulk/Gemfile"
2021-07-26 03:54:25.267 +0000 [INFO] (0001:transaction): Gem's home and path are being cleared.
2021-07-26 03:54:29.686 +0000 [INFO] (0001:transaction): Loaded plugin embulk-input-sqlserver (0.9.3)
2021-07-26 03:54:34.827 +0000 [INFO] (0001:transaction): Loaded plugin embulk-output-bigquery (0.4.10)
2021-07-26 03:54:34.915 +0000 [INFO] (0001:transaction): Using jTDS Driver
2021-07-26 03:54:34.919 +0000 [INFO] (0001:transaction): Connecting to jdbc:jtds:sqlserver://<ip-address>:1433/<database> options {appName=embulk-input-sqlserver, user=<user>, password=***, loginTimeout=300, socketTimeout=1800}
2021-07-26 03:54:35.050 +0000 [INFO] (0001:transaction): Using JDBC Driver 1.3.1
2021-07-26 03:54:35.166 +0000 [INFO] (0001:transaction): Using local thread executor with max_threads=64 / output tasks 32 = input tasks 1 * 32
2021-07-26 03:54:35.374 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Create dataset... <gcp-project>:<database> in us/eu
2021-07-26 03:54:36.781 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Create table... <gcp-project>:<database>.LOAD_TEMP_223c1748_452c_4688_bd1f_4c97343dffc6_<table>
2021-07-26 03:54:37.261 +0000 [INFO] (0001:transaction): {done: 0 / 1, running: 0}
2021-07-26 03:54:37.431 +0000 [INFO] (0056:task-0000): Using jTDS Driver
2021-07-26 03:54:37.432 +0000 [INFO] (0056:task-0000): Connecting to jdbc:jtds:sqlserver://<ip-address>:1433/<database> options {appName=embulk-input-sqlserver, user=<user>, password=***, loginTimeout=300, socketTimeout=1800}
2021-07-26 03:54:37.451 +0000 [INFO] (0056:task-0000): SQL: SELECT * FROM <table> WITH(NOLOCK)
2021-07-26 03:54:37.463 +0000 [INFO] (0056:task-0000): > 0.01 seconds
2021-07-26 03:54:37.504 +0000 [INFO] (embulk-output-executor-0): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2004.jsonl
2021-07-26 03:54:37.531 +0000 [INFO] (embulk-output-executor-1): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2006.jsonl
2021-07-26 03:54:37.555 +0000 [INFO] (0056:task-0000): Fetched 500 rows.
2021-07-26 03:54:37.556 +0000 [INFO] (embulk-output-executor-2): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2008.jsonl
2021-07-26 03:54:37.570 +0000 [INFO] (embulk-output-executor-3): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2010.jsonl
2021-07-26 03:54:37.575 +0000 [INFO] (embulk-output-executor-4): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2012.jsonl
2021-07-26 03:54:37.581 +0000 [INFO] (embulk-output-executor-5): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2014.jsonl
2021-07-26 03:54:37.588 +0000 [INFO] (embulk-output-executor-6): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2016.jsonl
2021-07-26 03:54:37.591 +0000 [INFO] (0056:task-0000): Fetched 1,000 rows.
2021-07-26 03:54:37.593 +0000 [INFO] (embulk-output-executor-7): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2018.jsonl
2021-07-26 03:54:37.598 +0000 [INFO] (embulk-output-executor-8): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2020.jsonl
2021-07-26 03:54:37.603 +0000 [INFO] (embulk-output-executor-9): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2022.jsonl
2021-07-26 03:54:37.608 +0000 [INFO] (embulk-output-executor-10): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2024.jsonl
2021-07-26 03:54:37.615 +0000 [INFO] (embulk-output-executor-11): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2026.jsonl
2021-07-26 03:54:37.617 +0000 [INFO] (embulk-output-executor-12): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2028.jsonl
2021-07-26 03:54:37.622 +0000 [INFO] (embulk-output-executor-13): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2030.jsonl
2021-07-26 03:54:37.624 +0000 [INFO] (embulk-output-executor-14): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2032.jsonl
2021-07-26 03:54:37.625 +0000 [INFO] (0056:task-0000): Fetched 2,000 rows.
2021-07-26 03:54:37.626 +0000 [INFO] (embulk-output-executor-15): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2034.jsonl
2021-07-26 03:54:37.630 +0000 [INFO] (embulk-output-executor-16): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2036.jsonl
2021-07-26 03:54:37.666 +0000 [INFO] (embulk-output-executor-18): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2038.jsonl
2021-07-26 03:54:37.669 +0000 [INFO] (embulk-output-executor-17): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2040.jsonl
2021-07-26 03:54:37.672 +0000 [INFO] (embulk-output-executor-19): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2042.jsonl
2021-07-26 03:54:37.676 +0000 [INFO] (embulk-output-executor-20): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2044.jsonl
2021-07-26 03:54:37.677 +0000 [INFO] (embulk-output-executor-21): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2046.jsonl
2021-07-26 03:54:37.683 +0000 [INFO] (embulk-output-executor-22): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2048.jsonl
2021-07-26 03:54:37.686 +0000 [INFO] (embulk-output-executor-23): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2050.jsonl
2021-07-26 03:54:37.688 +0000 [INFO] (embulk-output-executor-24): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2052.jsonl
2021-07-26 03:54:37.691 +0000 [INFO] (embulk-output-executor-25): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2054.jsonl
2021-07-26 03:54:37.694 +0000 [INFO] (embulk-output-executor-26): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2056.jsonl
2021-07-26 03:54:37.696 +0000 [INFO] (embulk-output-executor-27): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2058.jsonl
2021-07-26 03:54:37.698 +0000 [INFO] (embulk-output-executor-28): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2060.jsonl
2021-07-26 03:54:37.700 +0000 [INFO] (embulk-output-executor-29): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2062.jsonl
2021-07-26 03:54:37.702 +0000 [INFO] (0056:task-0000): Fetched 4,000 rows.
2021-07-26 03:54:37.702 +0000 [INFO] (embulk-output-executor-30): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2064.jsonl
2021-07-26 03:54:37.705 +0000 [INFO] (embulk-output-executor-31): embulk-output-bigquery: create /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2066.jsonl
2021-07-26 03:54:38.123 +0000 [INFO] (0056:task-0000): Fetched 8,000 rows.
2021-07-26 03:54:38.405 +0000 [INFO] (0056:task-0000): Fetched 16,000 rows.
2021-07-26 03:54:38.718 +0000 [INFO] (0056:task-0000): Fetched 32,000 rows.
2021-07-26 03:54:39.315 +0000 [INFO] (0056:task-0000): Fetched 64,000 rows.
2021-07-26 03:54:40.970 +0000 [INFO] (0056:task-0000): Fetched 128,000 rows.
2021-07-26 03:54:46.864 +0000 [INFO] (0056:task-0000): Fetched 256,000 rows.
2021-07-26 03:54:56.297 +0000 [INFO] (0056:task-0000): Fetched 512,000 rows.
2021-07-26 03:55:02.684 +0000 [INFO] (0056:task-0000): Fetched 1,024,000 rows.
2021-07-26 03:55:15.009 +0000 [INFO] (0056:task-0000): Fetched 2,048,000 rows.
2021-07-26 03:55:39.485 +0000 [INFO] (0056:task-0000): Fetched 4,096,000 rows.
2021-07-26 03:56:29.420 +0000 [INFO] (0056:task-0000): Fetched 8,192,000 rows.
2021-07-26 03:58:08.118 +0000 [INFO] (0056:task-0000): Fetched 16,384,000 rows.
2021-07-26 04:01:52.734 +0000 [INFO] (0056:task-0000): Fetched 32,768,000 rows.
2021-07-26 04:08:30.350 +0000 [INFO] (0056:task-0000): Fetched 65,536,000 rows.
2021-07-26 04:21:30.681 +0000 [INFO] (0056:task-0000): Fetched 131,072,000 rows.
2021-07-26 04:47:59.511 +0000 [INFO] (0056:task-0000): Fetched 262,144,000 rows.
2021-07-26 05:19:57.054 +0000 [INFO] (0001:transaction): {done: 1 / 1, running: 0}
2021-07-26 05:19:57.068 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2004.jsonl => <gcp-project>:gs://<gcp-project>/914d3a45-0fb8-46b2-bef2-c51bc2e41e91
2021-07-26 05:24:29.207 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2006.jsonl => <gcp-project>:gs://<gcp-project>/26228f6d-8874-4f23-b893-37a052c78a99
2021-07-26 05:28:59.301 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2008.jsonl => <gcp-project>:gs://<gcp-project>/5b661b8c-00ed-43f1-9f0f-02397f9d79ad
2021-07-26 05:33:32.375 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2010.jsonl => <gcp-project>:gs://<gcp-project>/b5ccd953-32e6-45b3-a5bf-766870d45da0
2021-07-26 05:38:05.113 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2012.jsonl => <gcp-project>:gs://<gcp-project>/f138d9c8-bbfe-49d2-9bfc-0ac8ba52e8d2
2021-07-26 05:43:01.649 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2014.jsonl => <gcp-project>:gs://<gcp-project>/0c007d22-b3b7-438b-943d-7818ff96e949
2021-07-26 05:46:01.737 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2016.jsonl => <gcp-project>:gs://<gcp-project>/9b33d287-8533-46be-9e72-894dd5ba48b2
2021-07-26 05:48:48.831 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2018.jsonl => <gcp-project>:gs://<gcp-project>/e621873d-3eae-40ee-9bf3-95e938c28752
2021-07-26 05:51:35.519 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2020.jsonl => <gcp-project>:gs://<gcp-project>/6b0cb6eb-0802-4369-b48b-da218032e5aa
2021-07-26 05:54:21.340 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2022.jsonl => <gcp-project>:gs://<gcp-project>/d281c8b0-f494-43ed-bd17-a879b75e428b
2021-07-26 05:57:05.311 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2024.jsonl => <gcp-project>:gs://<gcp-project>/f52a5bfd-e9ad-4365-9700-d9800275d5d7
2021-07-26 05:59:47.662 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2026.jsonl => <gcp-project>:gs://<gcp-project>/dd9c3243-015e-41d2-bf80-f0d393a63227
2021-07-26 06:02:28.147 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2028.jsonl => <gcp-project>:gs://<gcp-project>/77fc3702-4622-47f6-993a-fc6cf6c21aea
2021-07-26 06:05:08.972 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2030.jsonl => <gcp-project>:gs://<gcp-project>/8bce0fb6-c784-4589-b407-36173d13447e
2021-07-26 06:07:49.137 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2032.jsonl => <gcp-project>:gs://<gcp-project>/7a16b10f-e644-48cd-a296-8cc4050e7797
2021-07-26 06:10:29.571 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2034.jsonl => <gcp-project>:gs://<gcp-project>/5fa6d86b-5279-4a5a-ad11-b786040c5f6c
2021-07-26 06:13:12.003 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2036.jsonl => <gcp-project>:gs://<gcp-project>/d2c532e3-a097-4ca8-8b38-4ff1faf14dd0
2021-07-26 06:15:55.740 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2038.jsonl => <gcp-project>:gs://<gcp-project>/b17ab4bd-1050-4a68-aa2e-bc286a48b20d
2021-07-26 06:18:39.134 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2040.jsonl => <gcp-project>:gs://<gcp-project>/e7d95bbd-1e21-4b95-9d5e-c14a9041de9f
2021-07-26 06:21:47.724 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2042.jsonl => <gcp-project>:gs://<gcp-project>/0e68d5aa-b833-4845-b66f-17c315048101
2021-07-26 06:24:29.456 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2044.jsonl => <gcp-project>:gs://<gcp-project>/1789e555-3cf7-464b-9881-e2a911b0d9b0
2021-07-26 06:27:11.109 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2046.jsonl => <gcp-project>:gs://<gcp-project>/6193cece-5229-4d6f-934e-9904c419fe85
2021-07-26 06:29:53.558 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2048.jsonl => <gcp-project>:gs://<gcp-project>/a1acc584-aec4-4c7f-b84c-19f8ae69756c
2021-07-26 06:32:35.466 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2050.jsonl => <gcp-project>:gs://<gcp-project>/75a3b4cf-41ea-48f8-a91d-14e2a1259827
2021-07-26 06:35:17.247 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2052.jsonl => <gcp-project>:gs://<gcp-project>/0095deaf-f7b4-40ae-9a38-d304104e6de1
2021-07-26 06:37:59.073 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2054.jsonl => <gcp-project>:gs://<gcp-project>/9c3f22f5-fa07-4e56-ad83-3830e4752699
2021-07-26 06:40:40.169 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2056.jsonl => <gcp-project>:gs://<gcp-project>/6df3c144-524f-47c8-a937-8d6f3c19a834
2021-07-26 06:43:21.279 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2058.jsonl => <gcp-project>:gs://<gcp-project>/9b0427c6-87e0-4034-89d4-5a14e613df86
2021-07-26 06:46:02.571 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2060.jsonl => <gcp-project>:gs://<gcp-project>/f03ec231-4a85-4ddb-acc4-fdfed55573da
2021-07-26 06:48:43.657 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2062.jsonl => <gcp-project>:gs://<gcp-project>/0027a8f4-eb55-4732-b40f-25cf41b576e0
2021-07-26 06:51:24.725 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2064.jsonl => <gcp-project>:gs://<gcp-project>/8d16aa37-c893-4490-ac15-d87933904ead
2021-07-26 06:54:05.479 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Insert object... /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2066.jsonl => <gcp-project>:gs://<gcp-project>/3b7e0eac-a055-4d03-a3ad-4d52217a717e
2021-07-26 06:56:47.753 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job starting... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] ["gs://<gcp-project>/914d3a45-0fb8-46b2-bef2-c51bc2e41e91", "gs://<gcp-project>/26228f6d-8874-4f23-b893-37a052c78a99", "gs://<gcp-project>/5b661b8c-00ed-43f1-9f0f-02397f9d79ad", "gs://<gcp-project>/b5ccd953-32e6-45b3-a5bf-766870d45da0", "gs://<gcp-project>/f138d9c8-bbfe-49d2-9bfc-0ac8ba52e8d2", "gs://<gcp-project>/0c007d22-b3b7-438b-943d-7818ff96e949", "gs://<gcp-project>/9b33d287-8533-46be-9e72-894dd5ba48b2", "gs://<gcp-project>/e621873d-3eae-40ee-9bf3-95e938c28752", "gs://<gcp-project>/6b0cb6eb-0802-4369-b48b-da218032e5aa", "gs://<gcp-project>/d281c8b0-f494-43ed-bd17-a879b75e428b", "gs://<gcp-project>/f52a5bfd-e9ad-4365-9700-d9800275d5d7", "gs://<gcp-project>/dd9c3243-015e-41d2-bf80-f0d393a63227", "gs://<gcp-project>/77fc3702-4622-47f6-993a-fc6cf6c21aea", "gs://<gcp-project>/8bce0fb6-c784-4589-b407-36173d13447e", "gs://<gcp-project>/7a16b10f-e644-48cd-a296-8cc4050e7797", "gs://<gcp-project>/5fa6d86b-5279-4a5a-ad11-b786040c5f6c", "gs://<gcp-project>/d2c532e3-a097-4ca8-8b38-4ff1faf14dd0", "gs://<gcp-project>/b17ab4bd-1050-4a68-aa2e-bc286a48b20d", "gs://<gcp-project>/e7d95bbd-1e21-4b95-9d5e-c14a9041de9f", "gs://<gcp-project>/0e68d5aa-b833-4845-b66f-17c315048101", "gs://<gcp-project>/1789e555-3cf7-464b-9881-e2a911b0d9b0", "gs://<gcp-project>/6193cece-5229-4d6f-934e-9904c419fe85", "gs://<gcp-project>/a1acc584-aec4-4c7f-b84c-19f8ae69756c", "gs://<gcp-project>/75a3b4cf-41ea-48f8-a91d-14e2a1259827", "gs://<gcp-project>/0095deaf-f7b4-40ae-9a38-d304104e6de1", "gs://<gcp-project>/9c3f22f5-fa07-4e56-ad83-3830e4752699", "gs://<gcp-project>/6df3c144-524f-47c8-a937-8d6f3c19a834", "gs://<gcp-project>/9b0427c6-87e0-4034-89d4-5a14e613df86", "gs://<gcp-project>/f03ec231-4a85-4ddb-acc4-fdfed55573da", "gs://<gcp-project>/0027a8f4-eb55-4732-b40f-25cf41b576e0", "gs://<gcp-project>/8d16aa37-c893-4490-ac15-d87933904ead", "gs://<gcp-project>/3b7e0eac-a055-4d03-a3ad-4d52217a717e"] => <gcp-project>:<database>.LOAD_TEMP_223c1748_452c_4688_bd1f_4c97343dffc6_<table> in us/eu
2021-07-26 06:56:48.937 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job checking... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] elapsed_time:2.2e-05sec status:[RUNNING]
2021-07-26 06:56:59.316 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job checking... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] elapsed_time:10.38038sec status:[RUNNING]
2021-07-26 06:57:09.619 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job checking... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] elapsed_time:20.683221sec status:[RUNNING]
2021-07-26 06:57:19.886 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job checking... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] elapsed_time:30.951071sec status:[RUNNING]
2021-07-26 06:57:30.164 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job checking... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] elapsed_time:41.228837999999996sec status:[RUNNING]
2021-07-26 06:57:40.426 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job checking... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] elapsed_time:51.490259sec status:[RUNNING]
2021-07-26 06:57:50.806 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job checking... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] elapsed_time:61.870589sec status:[RUNNING]
2021-07-26 06:58:01.086 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job checking... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] elapsed_time:72.15026399999999sec status:[RUNNING]
2021-07-26 06:58:11.363 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job checking... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] elapsed_time:82.427911sec status:[RUNNING]
2021-07-26 06:58:21.628 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job checking... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] elapsed_time:92.692544sec status:[RUNNING]
2021-07-26 06:58:31.904 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job checking... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] elapsed_time:102.968209sec status:[RUNNING]
2021-07-26 06:58:42.192 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job checking... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] elapsed_time:113.25699sec status:[RUNNING]
2021-07-26 06:58:52.477 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job completed... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] elapsed_time:123.541477sec status:[DONE]
2021-07-26 06:58:52.478 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Load job response... job_id:[embulk_load_job_8e63aa75-7921-458d-a961-3dd541affb16] response.statistics:{:creation_time=>1627282608263, :completion_ratio=>1, :start_time=>1627282608751, :end_time=>1627282724525, :load=>{:input_file_bytes=>277072049347, :output_bytes=>81298975357, :output_rows=>418708680, :bad_records=>0, :input_files=>32}}
2021-07-26 06:58:52.480 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/914d3a45-0fb8-46b2-bef2-c51bc2e41e91
2021-07-26 06:58:52.874 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/26228f6d-8874-4f23-b893-37a052c78a99
2021-07-26 06:58:53.224 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/5b661b8c-00ed-43f1-9f0f-02397f9d79ad
2021-07-26 06:58:53.511 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/b5ccd953-32e6-45b3-a5bf-766870d45da0
2021-07-26 06:58:53.807 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/f138d9c8-bbfe-49d2-9bfc-0ac8ba52e8d2
2021-07-26 06:58:54.106 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/0c007d22-b3b7-438b-943d-7818ff96e949
2021-07-26 06:58:54.399 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/9b33d287-8533-46be-9e72-894dd5ba48b2
2021-07-26 06:58:54.686 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/e621873d-3eae-40ee-9bf3-95e938c28752
2021-07-26 06:58:54.977 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/6b0cb6eb-0802-4369-b48b-da218032e5aa
2021-07-26 06:58:55.268 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/d281c8b0-f494-43ed-bd17-a879b75e428b
2021-07-26 06:58:55.570 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/f52a5bfd-e9ad-4365-9700-d9800275d5d7
2021-07-26 06:58:55.852 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/dd9c3243-015e-41d2-bf80-f0d393a63227
2021-07-26 06:58:56.150 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/77fc3702-4622-47f6-993a-fc6cf6c21aea
2021-07-26 06:58:56.430 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/8bce0fb6-c784-4589-b407-36173d13447e
2021-07-26 06:58:56.719 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/7a16b10f-e644-48cd-a296-8cc4050e7797
2021-07-26 06:58:57.016 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/5fa6d86b-5279-4a5a-ad11-b786040c5f6c
2021-07-26 06:58:57.323 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/d2c532e3-a097-4ca8-8b38-4ff1faf14dd0
2021-07-26 06:58:57.617 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/b17ab4bd-1050-4a68-aa2e-bc286a48b20d
2021-07-26 06:58:57.895 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/e7d95bbd-1e21-4b95-9d5e-c14a9041de9f
2021-07-26 06:58:58.189 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/0e68d5aa-b833-4845-b66f-17c315048101
2021-07-26 06:58:58.469 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/1789e555-3cf7-464b-9881-e2a911b0d9b0
2021-07-26 06:58:58.900 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/6193cece-5229-4d6f-934e-9904c419fe85
2021-07-26 06:58:59.191 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/a1acc584-aec4-4c7f-b84c-19f8ae69756c
2021-07-26 06:58:59.477 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/75a3b4cf-41ea-48f8-a91d-14e2a1259827
2021-07-26 06:58:59.752 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/0095deaf-f7b4-40ae-9a38-d304104e6de1
2021-07-26 06:59:00.037 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/9c3f22f5-fa07-4e56-ad83-3830e4752699
2021-07-26 06:59:00.315 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/6df3c144-524f-47c8-a937-8d6f3c19a834
2021-07-26 06:59:00.583 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/9b0427c6-87e0-4034-89d4-5a14e613df86
2021-07-26 06:59:00.870 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/f03ec231-4a85-4ddb-acc4-fdfed55573da
2021-07-26 06:59:01.152 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/0027a8f4-eb55-4732-b40f-25cf41b576e0
2021-07-26 06:59:01.440 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/8d16aa37-c893-4490-ac15-d87933904ead
2021-07-26 06:59:01.725 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete object... <gcp-project>:gs://<gcp-project>/3b7e0eac-a055-4d03-a3ad-4d52217a717e
2021-07-26 06:59:02.004 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Get table... <gcp-project>:<database>.LOAD_TEMP_223c1748_452c_4688_bd1f_4c97343dffc6_<table>
2021-07-26 06:59:02.288 +0000 [INFO] (0001:transaction): embulk-output-bigquery: transaction_report: {"num_input_rows":418708680,"num_response_rows":418708680,"num_output_rows":418708680,"num_rejected_rows":0}
2021-07-26 06:59:02.290 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Copy job starting... job_id:[embulk_copy_job_efc215da-9d91-4a75-9bb9-29bd20ee4ccd] <gcp-project>:<database>.LOAD_TEMP_223c1748_452c_4688_bd1f_4c97343dffc6_<table> => <gcp-project>:<database>.<table>
2021-07-26 06:59:02.800 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Copy job checking... job_id:[embulk_copy_job_efc215da-9d91-4a75-9bb9-29bd20ee4ccd] elapsed_time:2.1e-05sec status:[RUNNING]
2021-07-26 06:59:13.068 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Copy job completed... job_id:[embulk_copy_job_efc215da-9d91-4a75-9bb9-29bd20ee4ccd] elapsed_time:10.268432sec status:[DONE]
2021-07-26 06:59:13.069 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Copy job response... job_id:[embulk_copy_job_efc215da-9d91-4a75-9bb9-29bd20ee4ccd] response.statistics:{:creation_time=>1627282742539, :start_time=>1627282742662, :end_time=>1627282744565}
2021-07-26 06:59:13.069 +0000 [INFO] (0001:transaction): embulk-output-bigquery: Delete table... <gcp-project>:<database>.LOAD_TEMP_223c1748_452c_4688_bd1f_4c97343dffc6_<table>
2021-07-26 06:59:13.414 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2004.jsonl
2021-07-26 06:59:13.633 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2006.jsonl
2021-07-26 06:59:13.835 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2008.jsonl
2021-07-26 06:59:14.049 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2010.jsonl
2021-07-26 06:59:14.240 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2012.jsonl
2021-07-26 06:59:14.442 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2014.jsonl
2021-07-26 06:59:14.638 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2016.jsonl
2021-07-26 06:59:14.843 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2018.jsonl
2021-07-26 06:59:15.039 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2020.jsonl
2021-07-26 06:59:15.236 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2022.jsonl
2021-07-26 06:59:15.440 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2024.jsonl
2021-07-26 06:59:15.639 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2026.jsonl
2021-07-26 06:59:15.835 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2028.jsonl
2021-07-26 06:59:16.030 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2030.jsonl
2021-07-26 06:59:16.230 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2032.jsonl
2021-07-26 06:59:16.417 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2034.jsonl
2021-07-26 06:59:16.602 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2036.jsonl
2021-07-26 06:59:16.792 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2038.jsonl
2021-07-26 06:59:16.989 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2040.jsonl
2021-07-26 06:59:17.181 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2042.jsonl
2021-07-26 06:59:17.384 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2044.jsonl
2021-07-26 06:59:17.567 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2046.jsonl
2021-07-26 06:59:17.767 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2048.jsonl
2021-07-26 06:59:17.968 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2050.jsonl
2021-07-26 06:59:18.167 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2052.jsonl
2021-07-26 06:59:18.355 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2054.jsonl
2021-07-26 06:59:18.823 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2056.jsonl
2021-07-26 06:59:19.452 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2058.jsonl
2021-07-26 06:59:20.077 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2060.jsonl
2021-07-26 06:59:20.711 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2062.jsonl
2021-07-26 06:59:21.350 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2064.jsonl
2021-07-26 06:59:21.988 +0000 [INFO] (0001:transaction): embulk-output-bigquery: delete /tmp/embulk_output_bigquery_20210726-7-1g22m1a.7.2066.jsonl
2021-07-26 06:59:22.648 +0000 [INFO] (main): Committed.
2021-07-26 06:59:22.648 +0000 [INFO] (main): Next config diff: {"in":{},"out":{}}
ref : https://twitter.com/csk_pos/status/1418400742801481729?s=20
Summary
when transfering large amount of data double release() calls. and can not transfer to BigQuery
Environment
OS version : Ubuntu 18.04.1
Java version : 1.8.0_111
Embulk version : Embulk v0.9.23
Embulk configuration (YAML) : config.yml
Error Meeage
Also related.
This error may relate to https://github.com/embulk/embulk/issues/1218