Unable to pass form data to my resource

ThibaultDelaune-pro commented 3 years ago

Hi, I try to make a POST request to a rest resource with this Concourse Resource and I have the following error:

http response code: 500
http response text: {"_embedded":{"errors":[{"message":"Error reading from class path resource [\"http://artefact.my-repo.com/artifactory/Maven/appsList/0.0.1/appsList-0.0.1.properties\"]","logref":"RuntimeException","_links":{"self":{"href":"/"}}}]}}
Traceback (most recent call last):
File "/opt/resource/out", line 106, in <module>
print(HTTPResource().run(os.path.basename(__file__), sys.stdin.read(), sys.argv[1:]))
File "/opt/resource/out", line 82, in run
status_code, text = self.cmd(command_argument, rendered_params)
File "/opt/resource/out", line 42, in cmd
raise Exception('Unexpected response {}'.format(response.status_code))
Exception: Unexpected response 500

When I try with curl it works perfectly

curl --location --request POST 'http://dataflow.my-platform/apps' \
--form 'uri="http://artefact.my-repo.com/artifactory/Maven/appsList/0.0.1/appsList-0.0.1.properties"'

Here is my config:

resource_types:
- name: http-api
  type: docker-image
  source:
    repository: aequitas/http-api-resource

resources:
- name: dataflow-rest-api
  type: http-api
  source:
    uri: http://dataflow.my-platform

jobs:
- name: load-apps-with-rest-api
  public: true
  plan:
  - put: dataflow-rest-api
    params:
      uri: http://dataflow.my-platform/apps
      method: POST
      debug: true
      form_data:
        uri: http://artefact.my-repo.com/artifactory/Maven/appsList/0.0.1/appsList-0.0.1.properties

And here, the API I try to call.

I don't see what I did wrong... Isn't that the way to pass form-data arguments?

aequitas commented 3 years ago

The values in form_data are converted into JSON https://github.com/aequitas/concourse-http-api-resource/blob/master/assets/resource.py#L33, so maybe that's why the API is returning a 500 on it?

ThibaultDelaune-pro commented 3 years ago

ok, so there is no way to make these kind of calls with this resource?

aequitas commented 3 years ago

Not in the current version. I'm happy to merge a pullrequest to add this functionality if you like to add it. Could be as simple as adding a boolean argument to not convert to JSON. But I can't invest the time into testing this right now myself.

ThibaultDelaune-pro commented 3 years ago

I tried, I'm not a developer but I clone your project and try to run the tests but they failed ...

Is it normal that the requests are made on http://127.0.0.1?

___________________________________ test_out ___________________________________
opt/resource-tests/test_invocation.py:18: in test_out
    subprocess.check_output('/opt/resource/out', input=json.dumps(data).encode())
        data       = {'source': {'uri': 'http://127.0.0.1:45557/status/200'}, 'version': {}}
        httpbin    = <pytest_httpbin.serve.Server object at 0x7fda761698e0>
usr/local/lib/python3.9/subprocess.py:424: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        kwargs     = {'input': b'{"source": {"uri": "http://127.0.0.1:45557/status/200"}, "version": {}}'}
        popenargs  = ('/opt/resource/out',)
        timeout    = None
...

aequitas commented 3 years ago

The tests run using a local httpbin server to not be dependant on external factors during testing. So this is normal.

aequitas commented 3 years ago

How are you invoking the tests? When I run them at my end using make test they run fine.

ThibaultDelaune-pro commented 3 years ago

docker build -t aequitas/http-resource-test -f Dockerfile.tdd .
docker run --rm aequitas/http-resource-test

aequitas commented 3 years ago

Should work. But without more information I can't tell what's wrong. The output you pasted is truncated so I can't see the error message itself.

ThibaultDelaune-pro commented 3 years ago

Here is the whole trace

$ docker run --rm aequitas/http-resource-test
============================= test session starts ==============================
platform linux -- Python 3.9.2, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /opt/resource-tests
plugins: httpbin-1.0.0, pylama-7.7.1
collected 8 items

opt/resource-tests/test_hipchat.py s                                     [ 12%]
opt/resource-tests/test_invocation.py FFFFF.F                            [100%]

=================================== FAILURES ===================================
___________________________________ test_out ___________________________________
opt/resource-tests/test_invocation.py:18: in test_out
    subprocess.check_output('/opt/resource/out', input=json.dumps(data).encode())
        data       = {'source': {'uri': 'http://127.0.0.1:45557/status/200'}, 'version': {}}
        httpbin    = <pytest_httpbin.serve.Server object at 0x7fda761698e0>
usr/local/lib/python3.9/subprocess.py:424: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        kwargs     = {'input': b'{"source": {"uri": "http://127.0.0.1:45557/status/200"}, "version": {}}'}
        popenargs  = ('/opt/resource/out',)
        timeout    = None
usr/local/lib/python3.9/subprocess.py:505: in run
    with Popen(*popenargs, **kwargs) as process:
        capture_output = False
        check      = True
        input      = b'{"source": {"uri": "http://127.0.0.1:45557/status/200"}, "version": {}}'
        kwargs     = {'stdin': -1, 'stdout': -1}
        popenargs  = ('/opt/resource/out',)
        timeout    = None
usr/local/lib/python3.9/subprocess.py:951: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
        args       = '/opt/resource/out'
        bufsize    = -1
        c2pread    = 14
        c2pwrite   = 15
        close_fds  = True
        creationflags = 0
        cwd        = None
        encoding   = None
        env        = None
        errors     = None
        errread    = -1
        errwrite   = -1
        executable = None
        extra_groups = None
        f          = <_io.BufferedReader name=14>
        gid        = None
        gids       = None
        group      = None
        p2cread    = 12
        p2cwrite   = 13
        pass_fds   = ()
        preexec_fn = None
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['/', 'o', 'p', 't', '/', 'r', 'e', 's', 'o', ...>
        shell      = False
        start_new_session = False
        startupinfo = None
        stderr     = None
        stdin      = -1
        stdout     = -1
        text       = None
        uid        = None
        umask      = -1
        universal_newlines = None
        user       = None
usr/local/lib/python3.9/subprocess.py:1823: in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
E   OSError: [Errno 8] Exec format error: '/opt/resource/out'
        args       = ['/opt/resource/out']
        c2pread    = 14
        c2pwrite   = 15
        child_exception_type = <class 'OSError'>
        child_exec_never_called = False
        close_fds  = True
        creationflags = 0
        cwd        = None
        env        = None
        env_list   = None
        err_filename = '/opt/resource/out'
        err_msg    = 'Exec format error'
        errno_num  = 8
        errpipe_data = bytearray(b'OSError:8:')
        errpipe_read = 16
        errpipe_write = 17
        errread    = -1
        errwrite   = -1
        exception_name = bytearray(b'OSError')
        executable = b'/opt/resource/out'
        executable_list = (b'/opt/resource/out',)
        fds_to_keep = {17}
        gid        = None
        gids       = None
        hex_errno  = bytearray(b'8')
        low_fds_to_close = []
        orig_executable = '/opt/resource/out'
        p2cread    = 12
        p2cwrite   = 13
        part       = b''
        pass_fds   = ()
        pid        = 8
        preexec_fn = None
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['/', 'o', 'p', 't', '/', 'r', 'e', 's', 'o', ...>
        shell      = False
        start_new_session = False
        startupinfo = None
        sts        = 65280
        uid        = None
        umask      = -1
_______________________________ test_out_failure _______________________________
opt/resource-tests/test_invocation.py:31: in test_out_failure
    subprocess.check_output('/opt/resource/out', input=json.dumps(data).encode())
        data       = {'source': {'uri': 'http://127.0.0.1:45557/status/404'}, 'version': {}}
        httpbin    = <pytest_httpbin.serve.Server object at 0x7fda761698e0>
usr/local/lib/python3.9/subprocess.py:424: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        kwargs     = {'input': b'{"source": {"uri": "http://127.0.0.1:45557/status/404"}, "version": {}}'}
        popenargs  = ('/opt/resource/out',)
        timeout    = None
usr/local/lib/python3.9/subprocess.py:505: in run
    with Popen(*popenargs, **kwargs) as process:
        capture_output = False
        check      = True
        input      = b'{"source": {"uri": "http://127.0.0.1:45557/status/404"}, "version": {}}'
        kwargs     = {'stdin': -1, 'stdout': -1}
        popenargs  = ('/opt/resource/out',)
        timeout    = None
usr/local/lib/python3.9/subprocess.py:951: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
        args       = '/opt/resource/out'
        bufsize    = -1
        c2pread    = 14
        c2pwrite   = 15
        close_fds  = True
        creationflags = 0
        cwd        = None
        encoding   = None
        env        = None
        errors     = None
        errread    = -1
        errwrite   = -1
        executable = None
        extra_groups = None
        f          = <_io.BufferedReader name=14>
        gid        = None
        gids       = None
        group      = None
        p2cread    = 12
        p2cwrite   = 13
        pass_fds   = ()
        preexec_fn = None
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['/', 'o', 'p', 't', '/', 'r', 'e', 's', 'o', ...>
        shell      = False
        start_new_session = False
        startupinfo = None
        stderr     = None
        stdin      = -1
        stdout     = -1
        text       = None
        uid        = None
        umask      = -1
        universal_newlines = None
        user       = None
usr/local/lib/python3.9/subprocess.py:1823: in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
E   OSError: [Errno 8] Exec format error: '/opt/resource/out'
        args       = ['/opt/resource/out']
        c2pread    = 14
        c2pwrite   = 15
        child_exception_type = <class 'OSError'>
        child_exec_never_called = False
        close_fds  = True
        creationflags = 0
        cwd        = None
        env        = None
        env_list   = None
        err_filename = '/opt/resource/out'
        err_msg    = 'Exec format error'
        errno_num  = 8
        errpipe_data = bytearray(b'OSError:8:')
        errpipe_read = 16
        errpipe_write = 17
        errread    = -1
        errwrite   = -1
        exception_name = bytearray(b'OSError')
        executable = b'/opt/resource/out'
        executable_list = (b'/opt/resource/out',)
        fds_to_keep = {17}
        gid        = None
        gids       = None
        hex_errno  = bytearray(b'8')
        low_fds_to_close = []
        orig_executable = '/opt/resource/out'
        p2cread    = 12
        p2cwrite   = 13
        part       = b''
        pass_fds   = ()
        pid        = 9
        preexec_fn = None
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['/', 'o', 'p', 't', '/', 'r', 'e', 's', 'o', ...>
        shell      = False
        start_new_session = False
        startupinfo = None
        sts        = 65280
        uid        = None
        umask      = -1
__________________________________ test_auth ___________________________________
opt/resource-tests/test_invocation.py:42: in test_auth
    subprocess.check_output('/opt/resource/out', input=json.dumps(data).encode())
        data       = {'source': {'uri': 'http://user:password@127.0.0.1:45557/basic-auth/user/password'}}
        httpbin    = <pytest_httpbin.serve.Server object at 0x7fda761698e0>
usr/local/lib/python3.9/subprocess.py:424: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        kwargs     = {'input': b'{"source": {"uri": "http://user:password@127.0.0.1:45557/basic-auth/user/password"}}'}
        popenargs  = ('/opt/resource/out',)
        timeout    = None
usr/local/lib/python3.9/subprocess.py:505: in run
    with Popen(*popenargs, **kwargs) as process:
        capture_output = False
        check      = True
        input      = b'{"source": {"uri": "http://user:password@127.0.0.1:45557/basic-auth/user/password"}}'
        kwargs     = {'stdin': -1, 'stdout': -1}
        popenargs  = ('/opt/resource/out',)
        timeout    = None
usr/local/lib/python3.9/subprocess.py:951: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
        args       = '/opt/resource/out'
        bufsize    = -1
        c2pread    = 14
        c2pwrite   = 15
        close_fds  = True
        creationflags = 0
        cwd        = None
        encoding   = None
        env        = None
        errors     = None
        errread    = -1
        errwrite   = -1
        executable = None
        extra_groups = None
        f          = <_io.BufferedReader name=14>
        gid        = None
        gids       = None
        group      = None
        p2cread    = 12
        p2cwrite   = 13
        pass_fds   = ()
        preexec_fn = None
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['/', 'o', 'p', 't', '/', 'r', 'e', 's', 'o', ...>
        shell      = False
        start_new_session = False
        startupinfo = None
        stderr     = None
        stdin      = -1
        stdout     = -1
        text       = None
        uid        = None
        umask      = -1
        universal_newlines = None
        user       = None
usr/local/lib/python3.9/subprocess.py:1823: in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
E   OSError: [Errno 8] Exec format error: '/opt/resource/out'
        args       = ['/opt/resource/out']
        c2pread    = 14
        c2pwrite   = 15
        child_exception_type = <class 'OSError'>
        child_exec_never_called = False
        close_fds  = True
        creationflags = 0
        cwd        = None
        env        = None
        env_list   = None
        err_filename = '/opt/resource/out'
        err_msg    = 'Exec format error'
        errno_num  = 8
        errpipe_data = bytearray(b'OSError:8:')
        errpipe_read = 16
        errpipe_write = 17
        errread    = -1
        errwrite   = -1
        exception_name = bytearray(b'OSError')
        executable = b'/opt/resource/out'
        executable_list = (b'/opt/resource/out',)
        fds_to_keep = {17}
        gid        = None
        gids       = None
        hex_errno  = bytearray(b'8')
        low_fds_to_close = []
        orig_executable = '/opt/resource/out'
        p2cread    = 12
        p2cwrite   = 13
        part       = b''
        pass_fds   = ()
        pid        = 10
        preexec_fn = None
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['/', 'o', 'p', 't', '/', 'r', 'e', 's', 'o', ...>
        shell      = False
        start_new_session = False
        startupinfo = None
        sts        = 65280
        uid        = None
        umask      = -1
__________________________________ test_json ___________________________________
opt/resource-tests/test_invocation.py:57: in test_json
    output = cmd('out', source)
        httpbin    = <pytest_httpbin.serve.Server object at 0x7fda761698e0>
        source     = {'json': {'test': 123}, 'method': 'POST', 'uri': 'http://127.0.0.1:45557/post', 'version': {}}
opt/resource-tests/helpers.py:24: in cmd
    output = subprocess.check_output(command, env=environment,
        args       = []
        cmd_name   = 'out'
        command    = ['/opt/resource/out']
        environment = {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ...}
        in_json    = '{"source": {"uri": "http://127.0.0.1:45557/post", "method": "POST", "json": {"test": 123}, "version": {}}, "version": {}, "params": {}}'
        params     = {}
        source     = {'json': {'test': 123}, 'method': 'POST', 'uri': 'http://127.0.0.1:45557/post', 'version': {}}
        version    = {}
usr/local/lib/python3.9/subprocess.py:424: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        kwargs     = {'env': {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ....ms": {}}', 'stderr': <_io.TextIOWrapper name="<_io.FileIO name=8 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>}
        popenargs  = (['/opt/resource/out'],)
        timeout    = None
usr/local/lib/python3.9/subprocess.py:505: in run
    with Popen(*popenargs, **kwargs) as process:
        capture_output = False
        check      = True
        input      = b'{"source": {"uri": "http://127.0.0.1:45557/post", "method": "POST", "json": {"test": 123}, "version": {}}, "version": {}, "params": {}}'
        kwargs     = {'env': {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ....extIOWrapper name="<_io.FileIO name=8 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>, 'stdin': -1, 'stdout': -1}
        popenargs  = (['/opt/resource/out'],)
        timeout    = None
usr/local/lib/python3.9/subprocess.py:951: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
        args       = ['/opt/resource/out']
        bufsize    = -1
        c2pread    = 14
        c2pwrite   = 15
        close_fds  = True
        creationflags = 0
        cwd        = None
        encoding   = None
        env        = {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ...}
        errors     = None
        errread    = -1
        errwrite   = 8
        executable = None
        extra_groups = None
        f          = <_io.BufferedReader name=14>
        gid        = None
        gids       = None
        group      = None
        p2cread    = 12
        p2cwrite   = 13
        pass_fds   = ()
        preexec_fn = None
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['/opt/resource/out']>
        shell      = False
        start_new_session = False
        startupinfo = None
        stderr     = <_io.TextIOWrapper name="<_io.FileIO name=8 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>
        stdin      = -1
        stdout     = -1
        text       = None
        uid        = None
        umask      = -1
        universal_newlines = None
        user       = None
usr/local/lib/python3.9/subprocess.py:1823: in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
E   OSError: [Errno 8] Exec format error: '/opt/resource/out'
        args       = ['/opt/resource/out']
        c2pread    = 14
        c2pwrite   = 15
        child_exception_type = <class 'OSError'>
        child_exec_never_called = False
        close_fds  = True
        creationflags = 0
        cwd        = None
        env        = {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ...}
        env_list   = [b'RESOURCE_DEBUG=1', b'HOSTNAME=ddeb6fae8593', b'PYTHON_PIP_VERSION=21.0.1', b'HOME=/root', b'GPG_KEY=E3FF2839C048B25...8', b'PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/b60e2320d9e8d02348525bd74e871e466afdf77c/get-pip.py', ...]
        err_filename = '/opt/resource/out'
        err_msg    = 'Exec format error'
        errno_num  = 8
        errpipe_data = bytearray(b'OSError:8:')
        errpipe_read = 16
        errpipe_write = 17
        errread    = -1
        errwrite   = 8
        exception_name = bytearray(b'OSError')
        executable = b'/opt/resource/out'
        executable_list = (b'/opt/resource/out',)
        fds_to_keep = {17}
        gid        = None
        gids       = None
        hex_errno  = bytearray(b'8')
        k          = b'TEST'
        low_fds_to_close = []
        orig_executable = '/opt/resource/out'
        p2cread    = 12
        p2cwrite   = 13
        part       = b''
        pass_fds   = ()
        pid        = 11
        preexec_fn = None
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['/opt/resource/out']>
        shell      = False
        start_new_session = False
        startupinfo = None
        sts        = 65280
        uid        = None
        umask      = -1
        v          = 'true'
______________________________ test_interpolation ______________________________
opt/resource-tests/test_invocation.py:79: in test_interpolation
    output = cmd('out', source)
        httpbin    = <pytest_httpbin.serve.Server object at 0x7fda761698e0>
        source     = {'json': {'array': ['{BUILD_NAME}'], 'object': {'test': '{BUILD_NAME}'}}, 'method': 'POST', 'uri': 'http://127.0.0.1:45557/post'}
opt/resource-tests/helpers.py:24: in cmd
    output = subprocess.check_output(command, env=environment,
        args       = []
        cmd_name   = 'out'
        command    = ['/opt/resource/out']
        environment = {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ...}
        in_json    = '{"source": {"uri": "http://127.0.0.1:45557/post", "method": "POST", "json": {"object": {"test": "{BUILD_NAME}"}, "array": ["{BUILD_NAME}"]}}, "version": {}, "params": {}}'
        params     = {}
        source     = {'json': {'array': ['{BUILD_NAME}'], 'object': {'test': '{BUILD_NAME}'}}, 'method': 'POST', 'uri': 'http://127.0.0.1:45557/post'}
        version    = {}
usr/local/lib/python3.9/subprocess.py:424: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        kwargs     = {'env': {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ....ms": {}}', 'stderr': <_io.TextIOWrapper name="<_io.FileIO name=8 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>}
        popenargs  = (['/opt/resource/out'],)
        timeout    = None
usr/local/lib/python3.9/subprocess.py:505: in run
    with Popen(*popenargs, **kwargs) as process:
        capture_output = False
        check      = True
        input      = b'{"source": {"uri": "http://127.0.0.1:45557/post", "method": "POST", "json": {"object": {"test": "{BUILD_NAME}"}, "array": ["{BUILD_NAME}"]}}, "version": {}, "params": {}}'
        kwargs     = {'env': {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ....extIOWrapper name="<_io.FileIO name=8 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>, 'stdin': -1, 'stdout': -1}
        popenargs  = (['/opt/resource/out'],)
        timeout    = None
usr/local/lib/python3.9/subprocess.py:951: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
        args       = ['/opt/resource/out']
        bufsize    = -1
        c2pread    = 14
        c2pwrite   = 15
        close_fds  = True
        creationflags = 0
        cwd        = None
        encoding   = None
        env        = {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ...}
        errors     = None
        errread    = -1
        errwrite   = 8
        executable = None
        extra_groups = None
        f          = <_io.BufferedReader name=14>
        gid        = None
        gids       = None
        group      = None
        p2cread    = 12
        p2cwrite   = 13
        pass_fds   = ()
        preexec_fn = None
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['/opt/resource/out']>
        shell      = False
        start_new_session = False
        startupinfo = None
        stderr     = <_io.TextIOWrapper name="<_io.FileIO name=8 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>
        stdin      = -1
        stdout     = -1
        text       = None
        uid        = None
        umask      = -1
        universal_newlines = None
        user       = None
usr/local/lib/python3.9/subprocess.py:1823: in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
E   OSError: [Errno 8] Exec format error: '/opt/resource/out'
        args       = ['/opt/resource/out']
        c2pread    = 14
        c2pwrite   = 15
        child_exception_type = <class 'OSError'>
        child_exec_never_called = False
        close_fds  = True
        creationflags = 0
        cwd        = None
        env        = {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ...}
        env_list   = [b'RESOURCE_DEBUG=1', b'HOSTNAME=ddeb6fae8593', b'PYTHON_PIP_VERSION=21.0.1', b'HOME=/root', b'GPG_KEY=E3FF2839C048B25...8', b'PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/b60e2320d9e8d02348525bd74e871e466afdf77c/get-pip.py', ...]
        err_filename = '/opt/resource/out'
        err_msg    = 'Exec format error'
        errno_num  = 8
        errpipe_data = bytearray(b'OSError:8:')
        errpipe_read = 16
        errpipe_write = 17
        errread    = -1
        errwrite   = 8
        exception_name = bytearray(b'OSError')
        executable = b'/opt/resource/out'
        executable_list = (b'/opt/resource/out',)
        fds_to_keep = {17}
        gid        = None
        gids       = None
        hex_errno  = bytearray(b'8')
        k          = b'TEST'
        low_fds_to_close = []
        orig_executable = '/opt/resource/out'
        p2cread    = 12
        p2cwrite   = 13
        part       = b''
        pass_fds   = ()
        pid        = 12
        preexec_fn = None
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['/opt/resource/out']>
        shell      = False
        start_new_session = False
        startupinfo = None
        sts        = 65280
        uid        = None
        umask      = -1
        v          = 'true'
_____________________________ test_data_urlencode ______________________________
opt/resource-tests/test_invocation.py:112: in test_data_urlencode
    output = cmd('out', source)
        httpbin    = <pytest_httpbin.serve.Server object at 0x7fda761698e0>
        source     = {'form_data': {'field': {'test': 123}}, 'method': 'POST', 'uri': 'http://127.0.0.1:45557/post'}
opt/resource-tests/helpers.py:24: in cmd
    output = subprocess.check_output(command, env=environment,
        args       = []
        cmd_name   = 'out'
        command    = ['/opt/resource/out']
        environment = {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ...}
        in_json    = '{"source": {"uri": "http://127.0.0.1:45557/post", "method": "POST", "form_data": {"field": {"test": 123}}}, "version": {}, "params": {}}'
        params     = {}
        source     = {'form_data': {'field': {'test': 123}}, 'method': 'POST', 'uri': 'http://127.0.0.1:45557/post'}
        version    = {}
usr/local/lib/python3.9/subprocess.py:424: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        kwargs     = {'env': {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ....ms": {}}', 'stderr': <_io.TextIOWrapper name="<_io.FileIO name=8 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>}
        popenargs  = (['/opt/resource/out'],)
        timeout    = None
usr/local/lib/python3.9/subprocess.py:505: in run
    with Popen(*popenargs, **kwargs) as process:
        capture_output = False
        check      = True
        input      = b'{"source": {"uri": "http://127.0.0.1:45557/post", "method": "POST", "form_data": {"field": {"test": 123}}}, "version": {}, "params": {}}'
        kwargs     = {'env': {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ....extIOWrapper name="<_io.FileIO name=8 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>, 'stdin': -1, 'stdout': -1}
        popenargs  = (['/opt/resource/out'],)
        timeout    = None
usr/local/lib/python3.9/subprocess.py:951: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
        args       = ['/opt/resource/out']
        bufsize    = -1
        c2pread    = 14
        c2pwrite   = 15
        close_fds  = True
        creationflags = 0
        cwd        = None
        encoding   = None
        env        = {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ...}
        errors     = None
        errread    = -1
        errwrite   = 8
        executable = None
        extra_groups = None
        f          = <_io.BufferedReader name=14>
        gid        = None
        gids       = None
        group      = None
        p2cread    = 12
        p2cwrite   = 13
        pass_fds   = ()
        preexec_fn = None
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['/opt/resource/out']>
        shell      = False
        start_new_session = False
        startupinfo = None
        stderr     = <_io.TextIOWrapper name="<_io.FileIO name=8 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>
        stdin      = -1
        stdout     = -1
        text       = None
        uid        = None
        umask      = -1
        universal_newlines = None
        user       = None
usr/local/lib/python3.9/subprocess.py:1823: in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
E   OSError: [Errno 8] Exec format error: '/opt/resource/out'
        args       = ['/opt/resource/out']
        c2pread    = 14
        c2pwrite   = 15
        child_exception_type = <class 'OSError'>
        child_exec_never_called = False
        close_fds  = True
        creationflags = 0
        cwd        = None
        env        = {'BUILD_ID': '123', 'BUILD_JOB_NAME': 'test-job', 'BUILD_NAME': '1', 'BUILD_PIPELINE_NAME': 'test-pipeline', ...}
        env_list   = [b'RESOURCE_DEBUG=1', b'HOSTNAME=ddeb6fae8593', b'PYTHON_PIP_VERSION=21.0.1', b'HOME=/root', b'GPG_KEY=E3FF2839C048B25...8', b'PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/b60e2320d9e8d02348525bd74e871e466afdf77c/get-pip.py', ...]
        err_filename = '/opt/resource/out'
        err_msg    = 'Exec format error'
        errno_num  = 8
        errpipe_data = bytearray(b'OSError:8:')
        errpipe_read = 16
        errpipe_write = 17
        errread    = -1
        errwrite   = 8
        exception_name = bytearray(b'OSError')
        executable = b'/opt/resource/out'
        executable_list = (b'/opt/resource/out',)
        fds_to_keep = {17}
        gid        = None
        gids       = None
        hex_errno  = bytearray(b'8')
        k          = b'TEST'
        low_fds_to_close = []
        orig_executable = '/opt/resource/out'
        p2cread    = 12
        p2cwrite   = 13
        part       = b''
        pass_fds   = ()
        pid        = 14
        preexec_fn = None
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['/opt/resource/out']>
        shell      = False
        start_new_session = False
        startupinfo = None
        sts        = 65280
        uid        = None
        umask      = -1
        v          = 'true'
=========================== short test summary info ============================
FAILED opt/resource-tests/test_invocation.py::test_out - OSError: [Errno 8] E...
FAILED opt/resource-tests/test_invocation.py::test_out_failure - OSError: [Er...
FAILED opt/resource-tests/test_invocation.py::test_auth - OSError: [Errno 8] ...
FAILED opt/resource-tests/test_invocation.py::test_json - OSError: [Errno 8] ...
FAILED opt/resource-tests/test_invocation.py::test_interpolation - OSError: [...
FAILED opt/resource-tests/test_invocation.py::test_data_urlencode - OSError: ...
==================== 6 failed, 1 passed, 1 skipped in 1.05s ====================

aequitas commented 3 years ago

Strange OSError: [Errno 8] Exec format error: '/opt/resource/out' is the real error here, but that would indicate a architecture or interpreter problem.

What kind of system are you running this on?

ThibaultDelaune-pro commented 3 years ago

Docker Desktop on windows 10 ... that may explain ...

aequitas commented 3 years ago

I've never tested this outside of macOS or Linux, I think Windows filesystem might not be playing nice with the symlink. Can you rename the resource.py file to replace the out symlink and try again? That might fix it.

ThibaultDelaune-pro commented 3 years ago

it worked!

aequitas / concourse-http-api-resource

Unable to pass form data to my resource #17