testcontainers / testcontainers-python

Testcontainers is a Python library that providing a friendly API to run Docker container. It is designed to create runtime environment to use during your automatic tests.
https://testcontainers-python.readthedocs.io/en/latest/
Apache License 2.0
1.53k stars 280 forks source link

feat: add copying files to/from container #676

Open mgorsk1 opened 1 month ago

mgorsk1 commented 1 month ago

solves https://github.com/testcontainers/testcontainers-python/issues/665

alexanderankin commented 1 month ago

i usually prefer using typing.Union[str, os.PathLike] - if you wanna use strings with this as well as paths

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 32.00000% with 17 lines in your changes missing coverage. Please review.

Please upload report for BASE (main@1e43923). Learn more about missing BASE report.

Files Patch % Lines
core/testcontainers/core/container.py 32.00% 16 Missing and 1 partial :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #676 +/- ## ======================================= Coverage ? 76.44% ======================================= Files ? 12 Lines ? 624 Branches ? 96 ======================================= Hits ? 477 Misses ? 120 Partials ? 27 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

alexanderankin commented 1 month ago

one other thing to consider is that there is this great interface "Transferable" in other language implementations-- does this PR make it harder or easier to add this to this implementation as well.

the interface (found here) is basically:

class Transferable(abc.ABC):
  @abstractmethod
  def transfer_to(content_stream, destination: typing.Union[str, os.PathLike]):
    ...

where content_stream is some bytesio or something pythonic for a place where you can write bytes to.

alexanderankin commented 1 month ago

when i tried to do it, i accidentally interpreted the interface a bit too widely and instead of passing just the content output stream, i passed the whole container as that first argument.

https://github.com/testcontainers/testcontainers-python/commit/4b7db74176c0f00ca275bc39409ebb891a93cbb9 - misc functions, next commit doesnt make sense without reading first

https://github.com/testcontainers/testcontainers-python/commit/bbc44ec8e6d0ce35d40466f1d636106d75e0b197 - the commit where i add the transferable stuff

alexanderankin commented 1 month ago

so im tiny bit hesitant to merge without a plan for adding transferable on top of this

mgorsk1 commented 1 month ago

do we need it to be 1:1 with java implementation though? any binary object now can be passed here using tempfile.NamedTemporaryObject:

with tempfile.NamedTemporaryObject() as f:

    f.write(binary_data)
    DockerContainer("nginx").with_copy_file_to_container(f.name, "/tmp/target.data").start()
alexanderankin commented 1 month ago

we dont need to be 1-1 but what if instead of a file you had a string? i think it is beneficial to not force people to create temporary files.

mgorsk1 commented 1 month ago

refactored for Transferable although creating temp files cannot be avoided, now instead of asking users for 3 extra lines we add some complexity to codebase here. working example

input:

from pathlib import Path

from testcontainers.core.container import DockerContainer, Transferable
from testcontainers.core.network import Network

c = DockerContainer('nginx')

network = Network()

c \
    .with_copy_file_to_container(Transferable(b'asd', Path('/tmp/mariusz.md1'))) \
    .with_copy_file_to_container(Transferable(Path('/Users/mariusz/Documents/Programming/testcontainers-python/README.md'),Path('/tmp/mariusz.md2'))) \
    .start()

c2 = DockerContainer('nginx')

print(c._container.exec_run('head /tmp/mariusz.md1'))
print(c._container.exec_run('head /tmp/mariusz.md2'))

output

ExecResult(exit_code=0, output=b'asd')
ExecResult(exit_code=0, output=b'[![Poetry](https://img.shields.io/endpoint?url=https://python-poetry.org/badge/v0.json)](https://python-poetry.org/)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n![PyPI - Version](https://img.shields.io/pypi/v/testcontainers)\n[![PyPI - License](https://img.shields.io/pypi/l/testcontainers.svg)](https://github.com/testcontainers/testcontainers-python/blob/main/LICENSE)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/testcontainers.svg)](https://pypi.python.org/pypi/testcontainers)\n[![codecov](https://codecov.io/gh/testcontainers/testcontainers-python/branch/master/graph/badge.svg)](https://codecov.io/gh/testcontainers/testcontainers-python)\n![Core Tests](https://github.com/testcontainers/testcontainers-python/actions/workflows/ci-core.yml/badge.svg)\n![Community Tests](https://github.com/testcontainers/testcontainers-python/actions/workflows/ci-community.yml/badge.svg)\n[![Docs](https://readthedocs.org/projects/testcontainers-python/badge/?version=latest)](http://testcontainers-python.readthedocs.io/en/latest/?badge=latest)\n\n')
Lenormju commented 1 month ago

@mgorsk1 I have opened #677 to solve the same issue (I did not see you already had opened one 😓). Yours have the Transferable feature. But I have the read_only feature. And they are not implemented the same way :

Are our two approaches complementary ? Reading again the Java documentation for copying files, there does not seem to be a difference, at the API level at least, between copying to a container before or after its startup.

mgorsk1 commented 1 month ago

I think copying before can already be solved by with_volume_mapping 😅

alexanderankin commented 1 month ago

committed some code here - https://github.com/testcontainers/testcontainers-python/compare/main...feat/665-with-copy-file-to-container

alexanderankin commented 1 month ago

rebased here https://github.com/testcontainers/testcontainers-python/compare/feat/665-with-copy-file-to-container_rebased

alexanderankin commented 1 month ago

since this is not a high priority feature this can probably wait for 1) more api refinement - feedback welcomed on my suggestions above 2) more of my free time, as I'm not sure ill be able to spend too much more time on this this week (just spent my entire week's time budget on this today).