ArchiveTeam / seesaw-kit

Making a reusable toolkit for writing seesaw scripts
Other
69 stars 30 forks source link

Docker not loading project. #122

Closed elit3ge closed 1 year ago

elit3ge commented 3 years ago
2021-05-13 09:06:28,442 - seesaw.warrior - DEBUG - Update warrior hq.
2021-05-13 09:06:28,442 - seesaw.warrior - DEBUG - Warrior ID ''.
2021-05-13 09:06:30,082 - seesaw.warrior - DEBUG - Select project pastebin
2021-05-13 09:06:30,083 - seesaw.warrior - DEBUG - Start selected project pastebin (reinstall=False)
2021-05-13 09:06:30,083 - seesaw.warrior - DEBUG - Install project pastebin
2021-05-13 09:06:30,083 - seesaw.warrior - DEBUG - git clone
2021-05-13 09:06:36,650 - seesaw.warrior - DEBUG - git operation: Cloning into '/home/warrior/projects/pastebin'...

2021-05-13 09:06:36,651 - seesaw.warrior - DEBUG - Install complete Cloning into '/home/warrior/projects/pastebin'...

2021-05-13 09:06:36,653 - seesaw.warrior - DEBUG - Result of the install process: True
2021-05-13 09:06:36,653 - seesaw.warrior - DEBUG - Clone project pastebin /home/warrior/projects/pastebin
2021-05-13 09:06:36,687 - seesaw.warrior - DEBUG - Cloning version aa87941
2021-05-13 09:06:36,822 - seesaw.warrior - DEBUG - Load pipeline /home/warrior/data/projects/pastebin-aa87941/pipeline.py
2021-05-13 09:06:36,824 - seesaw.warrior - DEBUG - Pipeline has been read. Begin ConfigValue collection
2021-05-13 09:06:36,824 - seesaw.warrior - DEBUG - Executing pipeline
2021-05-13 09:06:37,830 - seesaw.warrior - ERROR - Error loading pipeline
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 736, in start_selected_project
    (project, pipeline, config_values) = self.load_pipeline(
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 686, in load_pipeline
    exec(pipeline_str, local_context, global_context)
  File "<string>", line 52, in <module>
Exception: No usable Wget+At found.
2021-05-13 09:06:37,831 - seesaw.warrior - WARNING - Project pastebin did not install correctly and we're ignoring this problem.
rj-d2 commented 3 years ago

hi im getting the same error for reddit,

2021-06-04 10:18:19,716 - seesaw.warrior - WARNING - Project reddit did not install correctly and we're ignoring this problem. i did a fresh install, here is my docker-compose

version: '2.4'
services:
  archivewarrior:
    image: atdr.meo.ws/archiveteam/warrior-dockerfile:latest
    container_name: archive-warrior
    environment:
      - TZ=Europe/Amsterdam
      - SELECTED_PROJECT=auto
    ports:
      - 8001:8001
    restart: unless-stopped
k00lthing commented 3 years ago

same for GitHub

2021-08-18 19:30:58,001 - seesaw.warrior - DEBUG - Update warrior hq.
2021-08-18 19:30:58,001 - seesaw.warrior - DEBUG - Warrior ID ''.
2021-08-18 19:31:32,885 - seesaw.warrior - DEBUG - Select project githubtest2
2021-08-18 19:31:32,885 - seesaw.warrior - DEBUG - Start selected project githubtest2 (reinstall=False)
2021-08-18 19:31:32,885 - seesaw.warrior - DEBUG - Install project githubtest2
2021-08-18 19:31:32,885 - seesaw.warrior - DEBUG - git clone
2021-08-18 19:31:33,600 - seesaw.warrior - DEBUG - git operation: Cloning into '/home/warrior/projects/githubtest2'...

2021-08-18 19:31:33,601 - seesaw.warrior - DEBUG - Install complete Cloning into '/home/warrior/projects/githubtest2'...

2021-08-18 19:31:33,602 - seesaw.warrior - DEBUG - Result of the install process: True
2021-08-18 19:31:33,603 - seesaw.warrior - DEBUG - Clone project githubtest2 /home/warrior/projects/githubtest2
2021-08-18 19:31:33,615 - seesaw.warrior - DEBUG - Cloning version d5ed864
2021-08-18 19:31:33,631 - seesaw.warrior - DEBUG - Load pipeline /home/warrior/data/projects/githubtest2-d5ed864/pipeline.py
2021-08-18 19:31:33,632 - seesaw.warrior - DEBUG - Pipeline has been read. Begin ConfigValue collection
2021-08-18 19:31:33,632 - seesaw.warrior - DEBUG - Executing pipeline
2021-08-18 19:31:33,697 - seesaw.warrior - ERROR - Error loading pipeline
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 736, in start_selected_project
    (project, pipeline, config_values) = self.load_pipeline(
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 686, in load_pipeline
    exec(pipeline_str, local_context, global_context)
  File "<string>", line 51, in <module>
Exception: No usable Wget+At found.
2021-08-18 19:31:33,698 - seesaw.warrior - WARNING - Project githubtest2 did not install correctly and we're ignoring this problem.
nvahalik commented 2 years ago

At the top of my logs I see:

Looking for Wget+AT in ./wget-at
./wget-at: [Errno 2] No such file or directory: './wget-at'
Looking for Wget+AT in /home/warrior/data/wget-at-gnutls
/home/warrior/data/wget-at-gnutls: Incorrect Wget+AT version (want ['GNU Wget 1.20.3-at.20211001.01']).
samteezy commented 2 years ago

I'm seeing this as well on some projects like Reddit and githubtest2. Manually switching to URLTeam2 works properly, but there aren't any free items.

Traceback (most recent call last):
2022-01-04 16:10:02,889 - seesaw.warrior - ERROR - Error loading pipeline
2022-01-04 16:10:02,769 - seesaw.warrior - DEBUG - Cloning version 8a3f8cd
2022-01-04 16:10:02,761 - seesaw.warrior - DEBUG - Clone project reddit /home/warrior/projects/reddit
2022-01-04 16:10:02,791 - seesaw.warrior - DEBUG - Load pipeline /home/warrior/data/projects/reddit-8a3f8cd/pipeline.py
2022-01-04 16:10:02,793 - seesaw.warrior - DEBUG - Executing pipeline
2022-01-04 16:10:02,793 - seesaw.warrior - DEBUG - Pipeline has been read. Begin ConfigValue collection
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 736, in start_selected_project
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 686, in load_pipeline
    (project, pipeline, config_values) = self.load_pipeline(
    exec(pipeline_str, local_context, global_context)
  File "<string>", line 54, in <module>
2022-01-04 16:10:02,908 - seesaw.warrior - WARNING - Project reddit did not install correctly and we're ignoring this problem.
Exception: No usable Wget+At found.
billsargent commented 2 years ago

I've had to do this in order to get things to install.

RUN apt-get install pkg-config automake autoconf2.69 texinfo autopoint liblua5.1-0 liblua5.1-0-dev gettext texi2html gperf -y \ && git clone --branch v1.20.3-at https://github.com/ArchiveTeam/wget-lua.git \ && cd wget-lua && ./bootstrap && ./configure --with-ssl=gnutls --disable-nls \ && make \ && make install \ && cp src/wget /usr/bin/wget-lua \ && chmod a+x /usr/bin/wget-lua \ && mkdir -p /home/warrior/data \ && cp /usr/bin/wget-lua /wget \ && cp /usr/bin/wget-lua /home/warrior/data/wget-at-gnutls \ && cp /usr/bin/wget-lua /wget-at \ && cp /usr/bin/wget-lua /home/warrior/data/wget-at

It needs to be that branch of wget-at for most projects. I'm on a raspberry pi though. so mabe this doesn't affect you. I'm trying to work out the lack of proper support and priority of pi. Since nobody seems to care.

I just modified my Dockerfile for this.

iFloris commented 2 years ago

@billsargent could you put your dockerfile somewhere so we can use it? I am also having this problem with all my warriors and it is a shame I cannot contribute due to this bug.

See also https://github.com/ArchiveTeam/warrior-dockerfile/issues/62

TheTechRobo commented 2 years ago

@iFloris - the command they attached is a docker RUN command. That means you just add it to your Dockerfile.

iFloris commented 2 years ago

@iFloris - the command they attached is a docker RUN command. That means you just add it to your Dockerfile.

Ok, thanks! Adding it to my local dockerfile it is 🙂

billsargent commented 2 years ago

@iFloris - the command they attached is a docker RUN command. That means you just add it to your Dockerfile.

Ok, thanks! Adding it to my local dockerfile it is 🙂

You can also checkout my own repo I made ffor my own purposes.. https://github.com/billsargent/at-warrior-rpi

mindcombatant commented 2 years ago

for what it's worth, this issue used to affect me, and then i realized i was using an old version from dockerhub instead of the official image, listed here, which is atdr.meo.ws/archiveteam/warrior-dockerfile. switching to that image in my docker-run command fixed it.

billsargent commented 2 years ago

for what it's worth, this issue used to affect me, and then i realized i was using an old version from dockerhub instead of the official image, listed here, which is atdr.meo.ws/archiveteam/warrior-dockerfile. switching to that image in my docker-run command fixed it.

That was updated in the Dockerfile quite some time ago though.

Arkiver2 commented 1 year ago

This is not related to seesaw, but to https://github.com/ArchiveTeam/warrior-dockerfile/issues/62. Closing.