CI spawning check error for PR from fork: The provided API token is invalid

albertvillanova commented 1 year ago

See: https://github.com/huggingface/datasets-server/actions/runs/4796071992/jobs/8531405657?pr=1084#logs

______________________ test_real_check_spawning_response _______________________

app_config = AppConfig(assets=AssetsConfig(base_url='http://localhost/assets', storage_directory=None), cache=CacheConfig(mongo_dat...per_batch=1000, spawning_token='dummy_spawning_token', max_concurrent_requests_number=100, max_requests_per_second=50))

    @pytest.mark.asyncio
    async def test_real_check_spawning_response(app_config: AppConfig) -> None:
        semaphore = Semaphore(value=10)
        limiter = AsyncLimiter(10, time_period=1)

        headers = {"Authorization": f"API {CI_SPAWNING_TOKEN}"}
        async with ClientSession(headers=headers) as session:
            image_url = "http://testurl.test/test_image.jpg"
            image_urls = [image_url]
            spawning_url = app_config.urls_scan.spawning_url
            spawning_response = await check_spawning(image_urls, session, semaphore, limiter, spawning_url)
            assert spawning_response and isinstance(spawning_response, dict)
>           assert spawning_response["urls"] and isinstance(spawning_response["urls"], list)
E           KeyError: 'urls'

tests/job_runners/split/test_opt_in_out_urls_scan_from_streaming.py:309: KeyError

where spawning_response is:

{'detail': 'The provided API token is invalid.'}

AndreaFrancis commented 1 year ago

I will try to run again, it shouldn't have failed since the token is configured in the secrets and is a valid one.

albertvillanova commented 1 year ago

I think the root cause is because the PR is made from a fork.

Current implementation does not allow contributions from open source community members, who necessarily use forks.

albertvillanova commented 1 year ago

Related to:

712