Open lujiaying opened 1 year ago
In the log, notice some lines:
[DEBUG] [amlb.runners.aws:07:10:52.759] Skipping upload of `/home/ec2-user/automlbenchmark/resources/benchmarks/example.yaml` to s3 bucket.
[DEBUG] [amlb.runners.aws:07:10:52.759] Skipping upload of `/home/ec2-user/automlbenchmark/user_dir/config.yaml` to s3 bucket.
[DEBUG] [amlb.runners.aws:07:10:52.759] Skipping upload of `/home/ec2-user/automlbenchmark/user_dir/frameworks.yaml` to s3 bucket.
...
[INFO] [amlb.runners.aws:18:21:36.357] Starting new EC2 instance with params: autogluon_v0.5.1_high_il0.001 example 1h8c -Xseed=auto
user_dir is not uploaded to S3, because user_dir
is in app_dir
, refer to
https://github.com/openml/automlbenchmark/blob/7729f0dcaa37b2d5c48ca7c63b66ddd5f729e786/amlb/runners/aws.py#L825-L828
https://github.com/openml/automlbenchmark/blob/7729f0dcaa37b2d5c48ca7c63b66ddd5f729e786/amlb/runners/aws.py#L842-L852
script_params== autogluon_v0.5.1_high_il0.001 example 1h8c -Xseed=auto
, missing -u user_dir
https://github.com/openml/automlbenchmark/blob/7729f0dcaa37b2d5c48ca7c63b66ddd5f729e786/amlb/runners/aws.py#L557-L558
Alright, able to get rid of these errors by python runbenchmark.py autogluon_v0.5.1_high_il0.001 example 1h8c -u user_dir -m aws
the price is to directly modify resources/config.yaml
and resources/frameworks.yaml
@lujiaying, you found the reason:
user_dir is not uploaded to S3, because user_dir is in app_dir
if you put your user_dir
outside the amlb
folder structure, it should work.
Personally, I use directly or a sub-folder of the default ~/.config/automlbenchmark
.
If you need to easily switch between many configurations, you could also have a folder next to the app_dir
with all your configs: as soon as the user_dir
is completely dissociated from app_dir
then it will upload files correctly
Thanks for the solution, @sebhrusen .
Was wondering the design philosophy behind this:
If user_dir
is inside app_dir
, I totally agree that there's no need to upload user_dir
to s3.
However, it would be sweet if created EC2 instance can run with -u user_dir
. Now it seems that they did not specify -u
flag, which lead to my situation.
I just ran into this issue as well, and it was quite counterintuitive the root cause. It would be very nice if this worked. If I have a branch of AMLB and want to have my own configs separated from the original configs, it is hard to run them on AWS. I first need to copy them outside of AMLB directory and specify the copied location. This isn't ideal and would be confusing to users who are not experts on how AMLB works.
Hi amazing automlbenchmark team,
Following the instruction, I have put my custom yaml files under
PROJ_ROOT/user_dir
,And in
user_dir/config.yaml
, I have set upaws-resource_files
according to HOWTOThen I ran a test on aws mode by
python runbenchmark.py autogluon_v0.5.1_high_il0.001 example 1h8c -u user_dir -m aws
Noteautogluon_v0.5.1_high_il0.001
is a custom framework added inuser_dir/frameworks.yaml
(I tested successfully on local mode).However, it seems that `'{user}/config.yaml', '{user}/frameworks.yaml' are NOT uploaded. Error message are like:
May I know if I mess up anything? Shall I not use user_dir, but just directly modify the default config.yaml and framework.yaml? Thank you for the help!