Captainia
commented
3 weeks ago Pasting codebuild logs
=================================== FAILURES ===================================
--
831 | _ test_safety_file_exists_and_is_valid[669063966089.dkr.ecr.us-west-2.amazonaws.com/pr-huggingface-pytorch-inference:2.3.0-transformers4.46.0-cpu-py311-ubuntu22.04-pr-4392] _
832 | [gw2] linux -- Python 3.8.0 /usr/local/bin/python
833 |
834 | image = '669063966089.dkr.ecr.us-west-2.amazonaws.com/pr-huggingface-pytorch-inference:2.3.0-transformers4.46.0-cpu-py311-ubuntu22.04-pr-4392'
835 |
836 | @pytest.mark.model("N/A")
837 | @pytest.mark.skipif(is_canary_context(), reason="Skipping test because it does not run on canary")
838 | def test_safety_file_exists_and_is_valid(image):
839 | """
840 | Checks if the image has a safety report at the desired location and fails if any of the
841 | packages in the report have failed the safety check.
842 |
843 | :param image: str, image uri
844 | """
845 | repo_name, image_tag = image.split("/")[-1].split(":")
846 | # Make sure this container name doesn't conflict with the safety check test container name
847 | container_name = f"{repo_name}-{image_tag}-safety-file"
848 | # Add null entrypoint to ensure command exits immediately
849 | run(
850 | f"docker run -id " f"--name {container_name} " f"--entrypoint='/bin/bash' " f"{image}",
851 | hide=True,
852 | warn=True,
853 | )
854 |
855 | try:
856 | # Check if file exists
857 | docker_exec_cmd = f"docker exec -i {container_name}"
858 | safety_file_check = run(f"{docker_exec_cmd} test -f {SAFETY_FILE}", warn=True, hide=True)
859 | assert safety_file_check.ok, f"Safety file existence test failed for {image}"
860 |
861 | file_content = run(f"{docker_exec_cmd} cat {SAFETY_FILE}", warn=True, hide=True)
862 | raw_scan_result = json.loads(file_content.stdout)
863 | safety_report_object = SafetyPythonEnvironmentVulnerabilityReport(report=raw_scan_result)
864 |
865 | # processing safety reports
866 | report_log_template = "SAFETY_REPORT ({status}) [pkg: {pkg}] [installed: {installed}] [vulnerabilities: {vulnerabilities}]"
867 | failed_count = 0
868 | for report_item in safety_report_object.report:
869 | if report_item.scan_status == "FAILED":
870 | failed_count += 1
871 | LOGGER.error(
872 | report_log_template.format(
873 | status="FAILED",
874 | pkg=report_item.package,
875 | installed=report_item.installed,
876 | vulnerabilities=[
877 | entry for entry in report_item.vulnerabilities if not entry.ignored
878 | ],
879 | )
880 | )
881 | > assert failed_count == 0, f"{failed_count} package/s failed safety test for {image} !!!"
882 | E AssertionError: 1 package/s failed safety test for 669063966089.dkr.ecr.us-west-2.amazonaws.com/pr-huggingface-pytorch-inference:2.3.0-transformers4.46.0-cpu-py311-ubuntu22.04-pr-4392 !!!
883 | E assert 1 == 0
884 |
885 | sanity/test_safety_report_file.py:107: AssertionError
JingyaHuang
Issue #3870
transformers: 4.46.1accelerate: 1.1.0torch: 2.3.0diffusers: 0.31.0peft: 0.13.2Note:
If merging this PR should also close the associated Issue, please also add that Issue # to the Linked Issues section on the right.
All PR's are checked weekly for staleness. This PR will be closed if not updated in 30 days.
Description
Tests run
NOTE: By default, docker builds are disabled. In order to build your container, please update dlc_developer_config.toml and specify the framework to build in "build_frameworks"
Confused on how to run tests? Try using the helper utility...
Assuming your remote is called `origin` (you can find out more with `git remote -v`)... - Run default builds and tests for a particular buildspec - also commits and pushes changes to remote; Example: `python src/prepare_dlc_dev_environment.py -b -cp origin` - Enable specific tests for a buildspec or set of buildspecs - also commits and pushes changes to remote; Example: `python src/prepare_dlc_dev_environment.py -b -t sanity_tests -cp origin` - Restore TOML file when ready to merge `python src/prepare_dlc_dev_environment.py -rcp origin`NOTE: If you are creating a PR for a new framework version, please ensure success of the standard, rc, and efa sagemaker remote tests by updating the dlc_developer_config.toml file:
Expand
- [ ] `sagemaker_remote_tests = true` - [ ] `sagemaker_efa_tests = true` - [ ] `sagemaker_rc_tests = true` **Additionally, please run the sagemaker local tests in at least one revision:** - [ ] `sagemaker_local_tests = true`Formatting
black -l 100on my code (formatting tool: https://black.readthedocs.io/en/stable/getting_started.html)DLC image/dockerfile
Builds to Execute
Expand
Fill out the template and click the checkbox of the builds you'd like to execute *Note: Replace withAdditional context
PR Checklist
Expand
- [ ] I've prepended PR tag with frameworks/job this applies to : [mxnet, tensorflow, pytorch] | [ei/neuron/graviton] | [build] | [test] | [benchmark] | [ec2, ecs, eks, sagemaker] - [ ] If the PR changes affects SM test, I've modified dlc_developer_config.toml in my PR branch by setting sagemaker_tests = true and efa_tests = true - [ ] If this PR changes existing code, the change fully backward compatible with pre-existing code. (Non backward-compatible changes need special approval.) - [ ] (If applicable) I've documented below the DLC image/dockerfile this relates to - [ ] (If applicable) I've documented below the tests I've run on the DLC image - [ ] (If applicable) I've reviewed the licenses of updated and new binaries and their dependencies to make sure all licenses are on the Apache Software Foundation Third Party License Policy Category A or Category B license list. See [https://www.apache.org/legal/resolved.html](https://www.apache.org/legal/resolved.html). - [ ] (If applicable) I've scanned the updated and new binaries to make sure they do not have vulnerabilities associated with them. #### NEURON/GRAVITON Testing Checklist * When creating a PR: - [ ] I've modified `dlc_developer_config.toml` in my PR branch by setting `neuron_mode = true` or `graviton_mode = true` #### Benchmark Testing Checklist * When creating a PR: - [ ] I've modified `dlc_developer_config.toml` in my PR branch by setting `ec2_benchmark_tests = true` or `sagemaker_benchmark_tests = true`Pytest Marker Checklist
Expand
- [ ] (If applicable) I have added the marker `@pytest.mark.model("By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.