Refactor/configure - Githubissues

rvermootenct commented 1 year ago

name: Pull Request about: Create a pull request to make a contribution labels:

IMPORTANT NOTICE: I read and understood the guidelines for contributions to the TRD. The contribution may qualify for being compensated by the TRD grant if approved by the maintainers.

This PR resolves the issue #634. The following steps were performed:

Analysis: Using the configure script is confusing as the input syntax changes from question to question, it is also confusing that it asks you the same question multiple times on a number of the questions
Solution: Make all the inputs follow the same basic syntax.
Implementation: Updated the code to expect input that follows this basic pattern 'PKH1:, 'PKH2':' Was also able to remove what I feel as unnecessary validations. It seems like a lot of the inputs where validated twice.
Performed tests: Ran the configurer on master and then on this branch and looked at the difference. Tried to break it.
Documentation: Make sure to document the added changes in a proper way (Readme, help section, documentation, comments in code if needed)
Check list:

Work effort: 8h

jdsika commented 1 year ago

Working for ghostnet as well now?

rvermootenct commented 1 year ago

Working for ghostnet as well now?

Yeah, working on ghostnet. https://ghostnet.tzkt.io/tz1iZ9LkpAhN8X1L6RpBtfy3wxpEWzFrXz8j/operations/

jdsika commented 1 year ago

why is the script trying to connect to a local node?

--------------------------------------------
TRD Organization: Copyright 2021, see contributors.csv
huseyinabanox@gmail.com
Please leave copyright information
--------------------------------------------
2023-01-14 11:36:15,317 - MainThread - INFO - Arguments Configuration = {
 "reward_data_provider": "tzkt",
 "network": "MAINNET",
 "base_directory": "~/pymnt",
 "node_endpoint": "http://127.0.0.1:8732",
 "signer_endpoint": "http://127.0.0.1:6732",
 "docker": false,
 "verbose": "on",
 "dry_run": false,
 "api_base_url": null,
 "log_file": "~\\pymnt\\logs\\app.log"
}
2023-01-14 11:36:17,371 - MainThread - ERROR - Error, request ->http://127.0.0.1:8732/chains/main/blocks/head/context/constants<-, params ->None<-,
---
Error, exception ->HTTPConnectionPool(host='127.0.0.1', port=8732): Max retries exceeded with url: /chains/main/blocks/head/context/constants (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001E6842B3100>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))<-
2023-01-14 11:36:19,416 - MainThread - ERROR - Error, request ->http://127.0.0.1:8732/chains/main/blocks/head/context/constants<-, params ->None<-,
---
Error, exception ->HTTPConnectionPool(host='127.0.0.1', port=8732): Max retries exceeded with url: /chains/main/blocks/head/context/constants (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001E6842DA7F0>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))<-
2023-01-14 11:36:20,435 - MainThread - INFO - {'proof_of_work_nonce_size': 8, 'nonce_length': 32, 'max_anon_ops_per_block': 132, 'max_operation_data_length': 32768, 'max_proposals_per_delegate': 20, 'max_micheline_node_count': 50000, 'max_micheline_bytes_limit': 50000, 'max_allowed_global_constants_depth': 10000, 'cache_layout_size': 3, 'michelson_maximum_type_size': 2001, 'sc_max_wrapped_proof_binary_size': 30000, 'sc_rollup_message_size_limit': 4096, 'preserved_cycles': 5, 'blocks_per_cycle': 8192, 'blocks_per_commitment': 64, 'nonce_revelation_threshold': 256, 'blocks_per_stake_snapshot': 512, 'cycles_per_voting_period': 5, 'hard_gas_limit_per_operation': '1040000', 'hard_gas_limit_per_block': '5200000', 'proof_of_work_threshold': '70368744177663', 'minimal_stake': '6000000000', 'vdf_difficulty': '8000000000', 'seed_nonce_revelation_tip': '125000', 'origination_size': 257, 'baking_reward_fixed_portion': '10000000', 'baking_reward_bonus_per_slot': '4286', 'endorsing_reward_per_slot': '2857', 'cost_per_byte': '250', 'hard_storage_limit_per_operation': '60000', 'quorum_min': 2000, 'quorum_max': 7000, 'min_proposal_quorum': 500, 'liquidity_baking_subsidy': '2500000', 'liquidity_baking_toggle_ema_threshold': 1000000000, 'max_operations_time_to_live': 120, 'minimal_block_delay': '30', 'delay_increment_per_round': '15', 'consensus_committee_size': 7000, 'consensus_threshold': 4667, 'minimal_participation_ratio': {'numerator': 2, 'denominator': 3}, 'max_slashing_period': 2, 'frozen_deposits_percentage': 10, 'double_baking_punishment': '640000000', 'ratio_of_frozen_deposits_slashed_per_double_endorsement': {'numerator': 1, 'denominator': 2}, 'cache_script_size': 100000000, 'cache_stake_distribution_cycles': 8, 'cache_sampler_state_cycles': 8, 'tx_rollup_enable': True, 'tx_rollup_origination_size': 4000, 'tx_rollup_hard_size_limit_per_inbox': 500000, 'tx_rollup_hard_size_limit_per_message': 5000, 'tx_rollup_max_withdrawals_per_batch': 15, 'tx_rollup_commitment_bond': '10000000000', 'tx_rollup_finality_period': 40000, 'tx_rollup_withdraw_period': 40000, 'tx_rollup_max_inboxes_count': 40100, 'tx_rollup_max_messages_per_inbox': 1010, 'tx_rollup_max_commitments_count': 80100, 'tx_rollup_cost_per_byte_ema_factor': 120, 'tx_rollup_max_ticket_payload_size': 2048, 'tx_rollup_rejection_max_proof_size': 30000, 'tx_rollup_sunset_level': 3473409, 'dal_parametric': {'feature_enable': False, 'number_of_slots': 256, 'number_of_shards': 2048, 'endorsement_lag': 1, 'availability_threshold': 50, 'slot_size': 1048576, 'redundancy_factor': 16, 'page_size': 4096}, 'sc_rollup_enable': False, 'sc_rollup_origination_size': 6314, 'sc_rollup_challenge_window_in_blocks': 20160, 'sc_rollup_max_number_of_messages_per_commitment_period': 300000000, 'sc_rollup_stake_amount': '10000000000', 'sc_rollup_commitment_period_in_blocks': 30, 'sc_rollup_max_lookahead_in_blocks': 30000, 'sc_rollup_max_active_outbox_levels': 20160, 'sc_rollup_max_outbox_messages_per_level': 100, 'sc_rollup_number_of_sections_in_dissection': 32, 'sc_rollup_timeout_period_in_blocks': 20160, 'sc_rollup_max_number_of_cemented_commitments': 5, 'zk_rollup_enable': False, 'zk_rollup_origination_size': 4000, 'zk_rollup_min_pending_to_process': 10}
Tezos Reward Distributor (TRD) Baker Configuration Tool is Starting
This application will help you configure TRD to manage payouts for your bakery. Press enter to continue >

Also I am seeing that the path to the log file is created incorrectly on Windows:

"log_file": "~\\pymnt\\logs\\app.log"

jdsika commented 1 year ago

Not clear on how to abort the setup. Look at what happens when I press "qq"+"Enter"

 >qq
raceback (most recent call last):
  File "C:\repo\tezos-reward-distributor\src\configure.py", line 527, in <module>
    main(args)
  File "C:\repo\tezos-reward-distributor\src\configure.py", line 453, in main
    input("{} >".format(messages["hello"])).strip()
  File "C:\Program Files\Python39\lib\encodings\cp1252.py", line 22, in decode
    def decode(self, input, final=False):
KeyboardInterrupt

jdsika commented 1 year ago

Also I am seeing that the path to the log file is created incorrectly on Windows:

"log_file": "~\\pymnt\\logs\\app.log"

Maybe this is actually correct and it is just confusing as the base directory has other separators:

"base_directory": "~/pymnt",

rvermootenct commented 1 year ago

why is the script trying to connect to a local node?

--------------------------------------------
TRD Organization: Copyright 2021, see contributors.csv
huseyinabanox@gmail.com
Please leave copyright information
--------------------------------------------
2023-01-14 11:36:15,317 - MainThread - INFO - Arguments Configuration = {
 "reward_data_provider": "tzkt",
 "network": "MAINNET",
 "base_directory": "~/pymnt",
 "node_endpoint": "http://127.0.0.1:8732",
 "signer_endpoint": "http://127.0.0.1:6732",
 "docker": false,
 "verbose": "on",
 "dry_run": false,
 "api_base_url": null,
 "log_file": "~\\pymnt\\logs\\app.log"
}
2023-01-14 11:36:17,371 - MainThread - ERROR - Error, request ->http://127.0.0.1:8732/chains/main/blocks/head/context/constants<-, params ->None<-,
---
Error, exception ->HTTPConnectionPool(host='127.0.0.1', port=8732): Max retries exceeded with url: /chains/main/blocks/head/context/constants (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001E6842B3100>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))<-
2023-01-14 11:36:19,416 - MainThread - ERROR - Error, request ->http://127.0.0.1:8732/chains/main/blocks/head/context/constants<-, params ->None<-,
---
Error, exception ->HTTPConnectionPool(host='127.0.0.1', port=8732): Max retries exceeded with url: /chains/main/blocks/head/context/constants (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001E6842DA7F0>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))<-
2023-01-14 11:36:20,435 - MainThread - INFO - {'proof_of_work_nonce_size': 8, 'nonce_length': 32, 'max_anon_ops_per_block': 132, 'max_operation_data_length': 32768, 'max_proposals_per_delegate': 20, 'max_micheline_node_count': 50000, 'max_micheline_bytes_limit': 50000, 'max_allowed_global_constants_depth': 10000, 'cache_layout_size': 3, 'michelson_maximum_type_size': 2001, 'sc_max_wrapped_proof_binary_size': 30000, 'sc_rollup_message_size_limit': 4096, 'preserved_cycles': 5, 'blocks_per_cycle': 8192, 'blocks_per_commitment': 64, 'nonce_revelation_threshold': 256, 'blocks_per_stake_snapshot': 512, 'cycles_per_voting_period': 5, 'hard_gas_limit_per_operation': '1040000', 'hard_gas_limit_per_block': '5200000', 'proof_of_work_threshold': '70368744177663', 'minimal_stake': '6000000000', 'vdf_difficulty': '8000000000', 'seed_nonce_revelation_tip': '125000', 'origination_size': 257, 'baking_reward_fixed_portion': '10000000', 'baking_reward_bonus_per_slot': '4286', 'endorsing_reward_per_slot': '2857', 'cost_per_byte': '250', 'hard_storage_limit_per_operation': '60000', 'quorum_min': 2000, 'quorum_max': 7000, 'min_proposal_quorum': 500, 'liquidity_baking_subsidy': '2500000', 'liquidity_baking_toggle_ema_threshold': 1000000000, 'max_operations_time_to_live': 120, 'minimal_block_delay': '30', 'delay_increment_per_round': '15', 'consensus_committee_size': 7000, 'consensus_threshold': 4667, 'minimal_participation_ratio': {'numerator': 2, 'denominator': 3}, 'max_slashing_period': 2, 'frozen_deposits_percentage': 10, 'double_baking_punishment': '640000000', 'ratio_of_frozen_deposits_slashed_per_double_endorsement': {'numerator': 1, 'denominator': 2}, 'cache_script_size': 100000000, 'cache_stake_distribution_cycles': 8, 'cache_sampler_state_cycles': 8, 'tx_rollup_enable': True, 'tx_rollup_origination_size': 4000, 'tx_rollup_hard_size_limit_per_inbox': 500000, 'tx_rollup_hard_size_limit_per_message': 5000, 'tx_rollup_max_withdrawals_per_batch': 15, 'tx_rollup_commitment_bond': '10000000000', 'tx_rollup_finality_period': 40000, 'tx_rollup_withdraw_period': 40000, 'tx_rollup_max_inboxes_count': 40100, 'tx_rollup_max_messages_per_inbox': 1010, 'tx_rollup_max_commitments_count': 80100, 'tx_rollup_cost_per_byte_ema_factor': 120, 'tx_rollup_max_ticket_payload_size': 2048, 'tx_rollup_rejection_max_proof_size': 30000, 'tx_rollup_sunset_level': 3473409, 'dal_parametric': {'feature_enable': False, 'number_of_slots': 256, 'number_of_shards': 2048, 'endorsement_lag': 1, 'availability_threshold': 50, 'slot_size': 1048576, 'redundancy_factor': 16, 'page_size': 4096}, 'sc_rollup_enable': False, 'sc_rollup_origination_size': 6314, 'sc_rollup_challenge_window_in_blocks': 20160, 'sc_rollup_max_number_of_messages_per_commitment_period': 300000000, 'sc_rollup_stake_amount': '10000000000', 'sc_rollup_commitment_period_in_blocks': 30, 'sc_rollup_max_lookahead_in_blocks': 30000, 'sc_rollup_max_active_outbox_levels': 20160, 'sc_rollup_max_outbox_messages_per_level': 100, 'sc_rollup_number_of_sections_in_dissection': 32, 'sc_rollup_timeout_period_in_blocks': 20160, 'sc_rollup_max_number_of_cemented_commitments': 5, 'zk_rollup_enable': False, 'zk_rollup_origination_size': 4000, 'zk_rollup_min_pending_to_process': 10}
Tezos Reward Distributor (TRD) Baker Configuration Tool is Starting
This application will help you configure TRD to manage payouts for your bakery. Press enter to continue >

Also I am seeing that the path to the log file is created incorrectly on Windows:

"log_file": "~\\pymnt\\logs\\app.log"

Can you confirm, but I believe on master it also tries to connect to a local node. It needs the node because it does some validations during the configuration phrase. @jdsika

rvermootenct commented 1 year ago

Not clear on how to abort the setup. Look at what happens when I press "qq"+"Enter"

 >qq
raceback (most recent call last):
  File "C:\repo\tezos-reward-distributor\src\configure.py", line 527, in <module>
    main(args)
  File "C:\repo\tezos-reward-distributor\src\configure.py", line 453, in main
    input("{} >".format(messages["hello"])).strip()
  File "C:\Program Files\Python39\lib\encodings\cp1252.py", line 22, in decode
    def decode(self, input, final=False):
KeyboardInterrupt

What about Ctrl + C? did qq work before?

jdsika commented 1 year ago

$ python src/configure.py
Traceback (most recent call last):
  File "C:\repos\tezos-reward-distributor\src\configure.py", line 9, in <module>
    import yaml
ModuleNotFoundError: No module named 'yaml'

pip install -r requirements.txt
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: requests>=2.27.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 2)) (2.27.1)
Requirement already satisfied: base58>=1.0.2 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 3)) (2.1.0)
Requirement already satisfied: pyyaml>=5.3.1 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 4)) (5.4.1)
Requirement already satisfied: fysom>=2.1.6 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 5)) (2.1.6)
Requirement already satisfied: parse>=1.19.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 6)) (1.19.0)
Requirement already satisfied: transitions>=0.8.11 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 7)) (0.8.11)
Requirement already satisfied: tweepy>=4.6.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 8)) (4.8.0)
Requirement already satisfied: pip>=22.0.4 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 9)) (22.3.1)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from requests>=2.27.0->-r requirements.txt (line 2)) (2021.5.30)
Requirement already satisfied: idna<4,>=2.5 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from requests>=2.27.0->-r requirements.txt (line 2)) (3.2)
Requirement already satisfied: charset-normalizer~=2.0.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from requests>=2.27.0->-r requirements.txt (line 2)) (2.0.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from requests>=2.27.0->-r requirements.txt (line 2)) (1.26.6)
Requirement already satisfied: six in c:\users\q413005\appdata\roaming\python\python37\site-packages (from transitions>=0.8.11->-r requirements.txt (line 7)) (1.13.0)
Requirement already satisfied: requests-oauthlib<2,>=1.2.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from tweepy>=4.6.0->-r requirements.txt (line 8)) (1.3.1)
Requirement already satisfied: oauthlib<4,>=3.2.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from tweepy>=4.6.0->-r requirements.txt (line 8)) (3.2.0)

 python --version
Python 3.10.4

jdsika commented 1 year ago

https://bobbyhadz.com/blog/python-no-module-named-yaml

jdsika commented 1 year ago

why is the script trying to connect to a local node?

--------------------------------------------
TRD Organization: Copyright 2021, see contributors.csv
huseyinabanox@gmail.com
Please leave copyright information
--------------------------------------------
2023-01-14 11:36:15,317 - MainThread - INFO - Arguments Configuration = {
 "reward_data_provider": "tzkt",
 "network": "MAINNET",
 "base_directory": "~/pymnt",
 "node_endpoint": "http://127.0.0.1:8732",
 "signer_endpoint": "http://127.0.0.1:6732",
 "docker": false,
 "verbose": "on",
 "dry_run": false,
 "api_base_url": null,
 "log_file": "~\\pymnt\\logs\\app.log"
}
2023-01-14 11:36:17,371 - MainThread - ERROR - Error, request ->http://127.0.0.1:8732/chains/main/blocks/head/context/constants<-, params ->None<-,
---
Error, exception ->HTTPConnectionPool(host='127.0.0.1', port=8732): Max retries exceeded with url: /chains/main/blocks/head/context/constants (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001E6842B3100>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))<-
2023-01-14 11:36:19,416 - MainThread - ERROR - Error, request ->http://127.0.0.1:8732/chains/main/blocks/head/context/constants<-, params ->None<-,
---
Error, exception ->HTTPConnectionPool(host='127.0.0.1', port=8732): Max retries exceeded with url: /chains/main/blocks/head/context/constants (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001E6842DA7F0>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))<-
2023-01-14 11:36:20,435 - MainThread - INFO - {'proof_of_work_nonce_size': 8, 'nonce_length': 32, 'max_anon_ops_per_block': 132, 'max_operation_data_length': 32768, 'max_proposals_per_delegate': 20, 'max_micheline_node_count': 50000, 'max_micheline_bytes_limit': 50000, 'max_allowed_global_constants_depth': 10000, 'cache_layout_size': 3, 'michelson_maximum_type_size': 2001, 'sc_max_wrapped_proof_binary_size': 30000, 'sc_rollup_message_size_limit': 4096, 'preserved_cycles': 5, 'blocks_per_cycle': 8192, 'blocks_per_commitment': 64, 'nonce_revelation_threshold': 256, 'blocks_per_stake_snapshot': 512, 'cycles_per_voting_period': 5, 'hard_gas_limit_per_operation': '1040000', 'hard_gas_limit_per_block': '5200000', 'proof_of_work_threshold': '70368744177663', 'minimal_stake': '6000000000', 'vdf_difficulty': '8000000000', 'seed_nonce_revelation_tip': '125000', 'origination_size': 257, 'baking_reward_fixed_portion': '10000000', 'baking_reward_bonus_per_slot': '4286', 'endorsing_reward_per_slot': '2857', 'cost_per_byte': '250', 'hard_storage_limit_per_operation': '60000', 'quorum_min': 2000, 'quorum_max': 7000, 'min_proposal_quorum': 500, 'liquidity_baking_subsidy': '2500000', 'liquidity_baking_toggle_ema_threshold': 1000000000, 'max_operations_time_to_live': 120, 'minimal_block_delay': '30', 'delay_increment_per_round': '15', 'consensus_committee_size': 7000, 'consensus_threshold': 4667, 'minimal_participation_ratio': {'numerator': 2, 'denominator': 3}, 'max_slashing_period': 2, 'frozen_deposits_percentage': 10, 'double_baking_punishment': '640000000', 'ratio_of_frozen_deposits_slashed_per_double_endorsement': {'numerator': 1, 'denominator': 2}, 'cache_script_size': 100000000, 'cache_stake_distribution_cycles': 8, 'cache_sampler_state_cycles': 8, 'tx_rollup_enable': True, 'tx_rollup_origination_size': 4000, 'tx_rollup_hard_size_limit_per_inbox': 500000, 'tx_rollup_hard_size_limit_per_message': 5000, 'tx_rollup_max_withdrawals_per_batch': 15, 'tx_rollup_commitment_bond': '10000000000', 'tx_rollup_finality_period': 40000, 'tx_rollup_withdraw_period': 40000, 'tx_rollup_max_inboxes_count': 40100, 'tx_rollup_max_messages_per_inbox': 1010, 'tx_rollup_max_commitments_count': 80100, 'tx_rollup_cost_per_byte_ema_factor': 120, 'tx_rollup_max_ticket_payload_size': 2048, 'tx_rollup_rejection_max_proof_size': 30000, 'tx_rollup_sunset_level': 3473409, 'dal_parametric': {'feature_enable': False, 'number_of_slots': 256, 'number_of_shards': 2048, 'endorsement_lag': 1, 'availability_threshold': 50, 'slot_size': 1048576, 'redundancy_factor': 16, 'page_size': 4096}, 'sc_rollup_enable': False, 'sc_rollup_origination_size': 6314, 'sc_rollup_challenge_window_in_blocks': 20160, 'sc_rollup_max_number_of_messages_per_commitment_period': 300000000, 'sc_rollup_stake_amount': '10000000000', 'sc_rollup_commitment_period_in_blocks': 30, 'sc_rollup_max_lookahead_in_blocks': 30000, 'sc_rollup_max_active_outbox_levels': 20160, 'sc_rollup_max_outbox_messages_per_level': 100, 'sc_rollup_number_of_sections_in_dissection': 32, 'sc_rollup_timeout_period_in_blocks': 20160, 'sc_rollup_max_number_of_cemented_commitments': 5, 'zk_rollup_enable': False, 'zk_rollup_origination_size': 4000, 'zk_rollup_min_pending_to_process': 10}
Tezos Reward Distributor (TRD) Baker Configuration Tool is Starting
This application will help you configure TRD to manage payouts for your bakery. Press enter to continue >

Also I am seeing that the path to the log file is created incorrectly on Windows: "log_file": "~\\pymnt\\logs\\app.log"

Can you confirm, but I believe on master it also tries to connect to a local node. It needs the node because it does some validations during the configuration phrase. @jdsika

yes, the connection to the local node is being done on master as well. The question is: Are all arguments necessary for the configuration.py script?

rvermootenct commented 1 year ago

$ python src/configure.py
Traceback (most recent call last):
  File "C:\repos\tezos-reward-distributor\src\configure.py", line 9, in <module>
    import yaml
ModuleNotFoundError: No module named 'yaml'

pip install -r requirements.txt
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: requests>=2.27.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 2)) (2.27.1)
Requirement already satisfied: base58>=1.0.2 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 3)) (2.1.0)
Requirement already satisfied: pyyaml>=5.3.1 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 4)) (5.4.1)
Requirement already satisfied: fysom>=2.1.6 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 5)) (2.1.6)
Requirement already satisfied: parse>=1.19.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 6)) (1.19.0)
Requirement already satisfied: transitions>=0.8.11 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 7)) (0.8.11)
Requirement already satisfied: tweepy>=4.6.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 8)) (4.8.0)
Requirement already satisfied: pip>=22.0.4 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 9)) (22.3.1)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from requests>=2.27.0->-r requirements.txt (line 2)) (2021.5.30)
Requirement already satisfied: idna<4,>=2.5 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from requests>=2.27.0->-r requirements.txt (line 2)) (3.2)
Requirement already satisfied: charset-normalizer~=2.0.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from requests>=2.27.0->-r requirements.txt (line 2)) (2.0.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from requests>=2.27.0->-r requirements.txt (line 2)) (1.26.6)
Requirement already satisfied: six in c:\users\q413005\appdata\roaming\python\python37\site-packages (from transitions>=0.8.11->-r requirements.txt (line 7)) (1.13.0)
Requirement already satisfied: requests-oauthlib<2,>=1.2.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from tweepy>=4.6.0->-r requirements.txt (line 8)) (1.3.1)
Requirement already satisfied: oauthlib<4,>=3.2.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from tweepy>=4.6.0->-r requirements.txt (line 8)) (3.2.0)

 python --version
Python 3.10.4

It seems there is some issue with your env. I can see pyyaml is installed: "Requirement already satisfied: pyyaml" Maybe we can have a quick chat and try figure it out?

rvermootenct commented 1 year ago

why is the script trying to connect to a local node?

--------------------------------------------
TRD Organization: Copyright 2021, see contributors.csv
huseyinabanox@gmail.com
Please leave copyright information
--------------------------------------------
2023-01-14 11:36:15,317 - MainThread - INFO - Arguments Configuration = {
 "reward_data_provider": "tzkt",
 "network": "MAINNET",
 "base_directory": "~/pymnt",
 "node_endpoint": "http://127.0.0.1:8732",
 "signer_endpoint": "http://127.0.0.1:6732",
 "docker": false,
 "verbose": "on",
 "dry_run": false,
 "api_base_url": null,
 "log_file": "~\\pymnt\\logs\\app.log"
}
2023-01-14 11:36:17,371 - MainThread - ERROR - Error, request ->http://127.0.0.1:8732/chains/main/blocks/head/context/constants<-, params ->None<-,
---
Error, exception ->HTTPConnectionPool(host='127.0.0.1', port=8732): Max retries exceeded with url: /chains/main/blocks/head/context/constants (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001E6842B3100>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))<-
2023-01-14 11:36:19,416 - MainThread - ERROR - Error, request ->http://127.0.0.1:8732/chains/main/blocks/head/context/constants<-, params ->None<-,
---
Error, exception ->HTTPConnectionPool(host='127.0.0.1', port=8732): Max retries exceeded with url: /chains/main/blocks/head/context/constants (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001E6842DA7F0>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))<-
2023-01-14 11:36:20,435 - MainThread - INFO - {'proof_of_work_nonce_size': 8, 'nonce_length': 32, 'max_anon_ops_per_block': 132, 'max_operation_data_length': 32768, 'max_proposals_per_delegate': 20, 'max_micheline_node_count': 50000, 'max_micheline_bytes_limit': 50000, 'max_allowed_global_constants_depth': 10000, 'cache_layout_size': 3, 'michelson_maximum_type_size': 2001, 'sc_max_wrapped_proof_binary_size': 30000, 'sc_rollup_message_size_limit': 4096, 'preserved_cycles': 5, 'blocks_per_cycle': 8192, 'blocks_per_commitment': 64, 'nonce_revelation_threshold': 256, 'blocks_per_stake_snapshot': 512, 'cycles_per_voting_period': 5, 'hard_gas_limit_per_operation': '1040000', 'hard_gas_limit_per_block': '5200000', 'proof_of_work_threshold': '70368744177663', 'minimal_stake': '6000000000', 'vdf_difficulty': '8000000000', 'seed_nonce_revelation_tip': '125000', 'origination_size': 257, 'baking_reward_fixed_portion': '10000000', 'baking_reward_bonus_per_slot': '4286', 'endorsing_reward_per_slot': '2857', 'cost_per_byte': '250', 'hard_storage_limit_per_operation': '60000', 'quorum_min': 2000, 'quorum_max': 7000, 'min_proposal_quorum': 500, 'liquidity_baking_subsidy': '2500000', 'liquidity_baking_toggle_ema_threshold': 1000000000, 'max_operations_time_to_live': 120, 'minimal_block_delay': '30', 'delay_increment_per_round': '15', 'consensus_committee_size': 7000, 'consensus_threshold': 4667, 'minimal_participation_ratio': {'numerator': 2, 'denominator': 3}, 'max_slashing_period': 2, 'frozen_deposits_percentage': 10, 'double_baking_punishment': '640000000', 'ratio_of_frozen_deposits_slashed_per_double_endorsement': {'numerator': 1, 'denominator': 2}, 'cache_script_size': 100000000, 'cache_stake_distribution_cycles': 8, 'cache_sampler_state_cycles': 8, 'tx_rollup_enable': True, 'tx_rollup_origination_size': 4000, 'tx_rollup_hard_size_limit_per_inbox': 500000, 'tx_rollup_hard_size_limit_per_message': 5000, 'tx_rollup_max_withdrawals_per_batch': 15, 'tx_rollup_commitment_bond': '10000000000', 'tx_rollup_finality_period': 40000, 'tx_rollup_withdraw_period': 40000, 'tx_rollup_max_inboxes_count': 40100, 'tx_rollup_max_messages_per_inbox': 1010, 'tx_rollup_max_commitments_count': 80100, 'tx_rollup_cost_per_byte_ema_factor': 120, 'tx_rollup_max_ticket_payload_size': 2048, 'tx_rollup_rejection_max_proof_size': 30000, 'tx_rollup_sunset_level': 3473409, 'dal_parametric': {'feature_enable': False, 'number_of_slots': 256, 'number_of_shards': 2048, 'endorsement_lag': 1, 'availability_threshold': 50, 'slot_size': 1048576, 'redundancy_factor': 16, 'page_size': 4096}, 'sc_rollup_enable': False, 'sc_rollup_origination_size': 6314, 'sc_rollup_challenge_window_in_blocks': 20160, 'sc_rollup_max_number_of_messages_per_commitment_period': 300000000, 'sc_rollup_stake_amount': '10000000000', 'sc_rollup_commitment_period_in_blocks': 30, 'sc_rollup_max_lookahead_in_blocks': 30000, 'sc_rollup_max_active_outbox_levels': 20160, 'sc_rollup_max_outbox_messages_per_level': 100, 'sc_rollup_number_of_sections_in_dissection': 32, 'sc_rollup_timeout_period_in_blocks': 20160, 'sc_rollup_max_number_of_cemented_commitments': 5, 'zk_rollup_enable': False, 'zk_rollup_origination_size': 4000, 'zk_rollup_min_pending_to_process': 10}
Tezos Reward Distributor (TRD) Baker Configuration Tool is Starting
This application will help you configure TRD to manage payouts for your bakery. Press enter to continue >

Also I am seeing that the path to the log file is created incorrectly on Windows: "log_file": "~\\pymnt\\logs\\app.log"

Can you confirm, but I believe on master it also tries to connect to a local node. It needs the node because it does some validations during the configuration phrase. @jdsika

yes, the connection to the local node is being done on master as well. The question is: Are all arguments necessary for the configuration.py script?

I can remove docker, dry, verbose and logfile. I could see some utility for verbose and docker but I think we could get rid of them as well?

jdsika commented 1 year ago

I can remove docker, dry, verbose and logfile. I could see some utility for verbose and docker but I think we could get rid of them as well?

Remove everything that is unused. I could imagine that verbose logging is used in some of the functions e..g. with node interaction. The configure script can be dockerized of course. I cannot imagine why we would need dry_run?

jdsika commented 1 year ago

Conflicts

jdsika commented 1 year ago

@nicolasochem is it correct to remove the docker argument here?

jdsika commented 1 year ago

$ python src/configure.py
Traceback (most recent call last):
  File "C:\repos\tezos-reward-distributor\src\configure.py", line 9, in <module>
    import yaml
ModuleNotFoundError: No module named 'yaml'

pip install -r requirements.txt
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: requests>=2.27.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 2)) (2.27.1)
Requirement already satisfied: base58>=1.0.2 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 3)) (2.1.0)
Requirement already satisfied: pyyaml>=5.3.1 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 4)) (5.4.1)
Requirement already satisfied: fysom>=2.1.6 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 5)) (2.1.6)
Requirement already satisfied: parse>=1.19.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 6)) (1.19.0)
Requirement already satisfied: transitions>=0.8.11 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 7)) (0.8.11)
Requirement already satisfied: tweepy>=4.6.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 8)) (4.8.0)
Requirement already satisfied: pip>=22.0.4 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from -r requirements.txt (line 9)) (22.3.1)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from requests>=2.27.0->-r requirements.txt (line 2)) (2021.5.30)
Requirement already satisfied: idna<4,>=2.5 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from requests>=2.27.0->-r requirements.txt (line 2)) (3.2)
Requirement already satisfied: charset-normalizer~=2.0.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from requests>=2.27.0->-r requirements.txt (line 2)) (2.0.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from requests>=2.27.0->-r requirements.txt (line 2)) (1.26.6)
Requirement already satisfied: six in c:\users\q413005\appdata\roaming\python\python37\site-packages (from transitions>=0.8.11->-r requirements.txt (line 7)) (1.13.0)
Requirement already satisfied: requests-oauthlib<2,>=1.2.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from tweepy>=4.6.0->-r requirements.txt (line 8)) (1.3.1)
Requirement already satisfied: oauthlib<4,>=3.2.0 in c:\users\q413005\appdata\roaming\python\python37\site-packages (from tweepy>=4.6.0->-r requirements.txt (line 8)) (3.2.0)

 python --version
Python 3.10.4

It seems there is some issue with your env. I can see pyyaml is installed: "Requirement already satisfied: pyyaml" Maybe we can have a quick chat and try figure it out?

I figured this out! It was an issue with two versions of python being installed on the system!

jdsika commented 1 year ago

2023-01-24 10:26:53,755 - MainThread - ERROR - Error, request ->http://127.0.0.1:8732/chains/main/blocks/head/context/constants<-, params ->None<-,
---
Error, exception ->HTTPConnectionPool(host='127.0.0.1', port=8732): Max retries exceeded with url: /chains/main/blocks/head/context/constants (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001CAFEE38B08>: Failed to establish a new connection: [WinError 10061] Es konnte keine Verbindung hergestellt werden, da der Zielcomputer die Verbindung verweigerte'))<-
2023-01-24 10:26:55,802 - MainThread - ERROR - Error, request ->http://127.0.0.1:8732/chains/main/blocks/head/context/constants<-, params ->None<-,
---
Error, exception ->HTTPConnectionPool(host='127.0.0.1', port=8732): Max retries exceeded with url: /chains/main/blocks/head/context/constants (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001CAFEE54048>: Failed to establish a new connection: [WinError 10061] Es konnte keine Verbindung hergestellt werden, da der Zielcomputer die Verbindung verweigerte'))<-
2023-01-24 10:26:55,803 - MainThread - INFO - Failed to get network configuration constants from a local node (http://127.0.0.1:8732).
2023-01-24 10:26:56,179 - MainThread - INFO - {'proof_of_work_nonce_size': 8, 'nonce_length': 32, 'max_anon_ops_per_block': 132, 'max_operation_data_length': 32768, 'max_proposals_per_delegate': 20, 'max_micheline_node_count': 50000, 'max_micheline_bytes_limit': 50000, 'max_allowed_global_constants_depth': 10000, 'cache_layout_size': 3, 'michelson_maximum_type_size': 2001, 'sc_max_wrapped_proof_binary_size': 30000, 'sc_rollup_message_size_limit': 4096, 'preserved_cycles': 5, 'blocks_per_cycle': 8192, 'blocks_per_commitment': 64, 'nonce_revelation_threshold': 256, 'blocks_per_stake_snapshot': 512, 'cycles_per_voting_period': 5, 'hard_gas_limit_per_operation': '1040000', 'hard_gas_limit_per_block': '5200000', 'proof_of_work_threshold': '70368744177663', 'minimal_stake': '6000000000', 'vdf_difficulty': '8000000000', 'seed_nonce_revelation_tip': '125000', 'origination_size': 257, 'baking_reward_fixed_portion': '10000000', 'baking_reward_bonus_per_slot': '4286', 'endorsing_reward_per_slot': '2857', 'cost_per_byte': '250', 'hard_storage_limit_per_operation': '60000', 'quorum_min': 2000, 'quorum_max': 7000, 'min_proposal_quorum': 500, 'liquidity_baking_subsidy': '2500000', 'liquidity_baking_toggle_ema_threshold': 1000000000, 'max_operations_time_to_live': 120, 'minimal_block_delay': '30', 'delay_increment_per_round': '15', 'consensus_committee_size': 7000, 'consensus_threshold': 4667, 'minimal_participation_ratio': {'numerator': 2, 'denominator': 3}, 'max_slashing_period': 2, 'frozen_deposits_percentage': 10, 'double_baking_punishment': '640000000', 'ratio_of_frozen_deposits_slashed_per_double_endorsement': {'numerator': 1, 'denominator': 2}, 'cache_script_size': 100000000, 'cache_stake_distribution_cycles': 8, 'cache_sampler_state_cycles': 8, 'tx_rollup_enable': True, 'tx_rollup_origination_size': 4000, 'tx_rollup_hard_size_limit_per_inbox': 500000, 'tx_rollup_hard_size_limit_per_message': 5000, 'tx_rollup_max_withdrawals_per_batch': 15, 'tx_rollup_commitment_bond': '10000000000', 'tx_rollup_finality_period': 40000, 'tx_rollup_withdraw_period': 40000, 'tx_rollup_max_inboxes_count': 40100, 'tx_rollup_max_messages_per_inbox': 1010, 'tx_rollup_max_commitments_count': 80100, 'tx_rollup_cost_per_byte_ema_factor': 120, 'tx_rollup_max_ticket_payload_size': 2048, 'tx_rollup_rejection_max_proof_size': 30000, 'tx_rollup_sunset_level': 3473409, 'dal_parametric': {'feature_enable': False, 'number_of_slots': 256, 'number_of_shards': 2048, 'endorsement_lag': 1, 'availability_threshold': 50, 'slot_size': 1048576, 'redundancy_factor': 16, 'page_size': 4096}, 'sc_rollup_enable': False, 'sc_rollup_origination_size': 6314, 'sc_rollup_challenge_window_in_blocks': 20160, 'sc_rollup_max_number_of_messages_per_commitment_period': 300000000, 'sc_rollup_stake_amount': '10000000000', 'sc_rollup_commitment_period_in_blocks': 30, 'sc_rollup_max_lookahead_in_blocks': 30000, 'sc_rollup_max_active_outbox_levels': 20160, 'sc_rollup_max_outbox_messages_per_level': 100, 'sc_rollup_number_of_sections_in_dissection': 32, 'sc_rollup_timeout_period_in_blocks': 20160, 'sc_rollup_max_number_of_cemented_commitments': 5, 'zk_rollup_enable': False, 'zk_rollup_origination_size': 4000, 'zk_rollup_min_pending_to_process': 10}

nicolasochem commented 1 year ago

@nicolasochem is it correct to remove the docker argument here?

After searching through the code, I don't see the point of the docker arg at all. It's used in tests and in statistics, but it does not appear to change the behavior of the program otherwise.

Thank you for the refactor @rvermootenct

tezos-reward-distributor-organization / tezos-reward-distributor

Refactor/configure #651