Unblock plenaries.json serialization

[sandervh14]

Guido reports:

Running SKIP_SLOW=true ./run-all.sh         results in a td-plenaries.json
INFO:root:Report files must be found at: /Users/guido/code/transparentdemocracy/voting-data/data/input/plenary/html/*.html.

Processing plenary reports...: 0it [00:00, ?it/s]
Processing plenary reports...: 0it [00:00, ?it/s]

Linking motions with proposals...: 0it [00:00, ?it/s]
Linking motions with proposals...: 0it [00:00, ?it/s]
Traceback (most recent call last):
  File "/Users/guido/code/transparentdemocracy/voting-data/venv/bin/td-plenaries-json", line 8, in <module>
    sys.exit(write_plenaries_json())
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/guido/code/transparentdemocracy/voting-data/transparentdemocracy/plenaries/serialization.py", line 162, in write_plenaries_json
    JsonSerializer().serialize_plenaries(plenaries)
  File "/Users/guido/code/transparentdemocracy/voting-data/transparentdemocracy/plenaries/serialization.py", line 99, in serialize_plenaries
    self._serialize_plenaries(plenaries, "plenaries.json")
  File "/Users/guido/code/transparentdemocracy/voting-data/transparentdemocracy/plenaries/serialization.py", line 114, in _serialize_plenaries
    raise ValueError("No plenaries occur with motion-proposal links. Run the motion_proposal_linker.py before "
ValueError: No plenaries occur with motion-proposal links. Run the motion_proposal_linker.py before serializing plenaries.

Not blocking for me. Just wanted to let you guys know

[karel1980]

Looks like the reports weren't downloaded, I'll document it or automate it

Op do 23 mei 2024 09:04 schreef Sander Vanden Hautte < @.***>:

Guido reports:

Running SKIP_SLOW=true ./run-all.sh results in a td-plenaries.json INFO:root:Report files must be found at: /Users/guido/code/transparentdemocracy/voting-data/data/input/plenary/html/*.html.

Processing plenary reports...: 0it [00:00, ?it/s] Processing plenary reports...: 0it [00:00, ?it/s]

Linking motions with proposals...: 0it [00:00, ?it/s] Linking motions with proposals...: 0it [00:00, ?it/s] Traceback (most recent call last): File "/Users/guido/code/transparentdemocracy/voting-data/venv/bin/td-plenaries-json", line 8, in sys.exit(write_plenaries_json()) ^^^^^^^^^^^^^^^^^^^^^^ File "/Users/guido/code/transparentdemocracy/voting-data/transparentdemocracy/plenaries/serialization.py", line 162, in write_plenaries_json JsonSerializer().serialize_plenaries(plenaries) File "/Users/guido/code/transparentdemocracy/voting-data/transparentdemocracy/plenaries/serialization.py", line 99, in serialize_plenaries self._serialize_plenaries(plenaries, "plenaries.json") File "/Users/guido/code/transparentdemocracy/voting-data/transparentdemocracy/plenaries/serialization.py", line 114, in _serialize_plenaries raise ValueError("No plenaries occur with motion-proposal links. Run the motion_proposal_linker.py before " ValueError: No plenaries occur with motion-proposal links. Run the motion_proposal_linker.py before serializing plenaries.

Not blocking for me. Just wanted to let you guys know

— Reply to this email directly, view it on GitHub https://github.com/transparentdemocracy/voting-data/issues/44, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABFLPMDJRE7WBZCQXQ7IELZDWIJVAVCNFSM6AAAAABIFAQUL6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGMYTEMJSGA4TQNY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[sandervh14]

No worries. I promised Guido I would check this one.

@GuidoDechamps this indeed happened if you did not have any plenary reports in the input folder to extract plenary JSON from. Follow the instructions at https://github.com/transparentdemocracy/voting-data/tree/main?tab=readme-ov-file#downloading-and-generating-data.

I've now made our code fail early on that, with a sensical explanation and referring to the above link.

@karel1980 I didn't think it worth to automate this already, takes more time to implement than just triggering the download scripts.