posit-dev / rsconnect-python

Command line interface for publishing to Posit Connect
https://docs.posit.co/rsconnect-python/
GNU General Public License v2.0
28 stars 21 forks source link

rsconnect write-manifest not excluding files on windows #285

Closed shapiromatron closed 1 year ago

shapiromatron commented 1 year ago

When running rsconnect write-manifest, the generated manifest.json still includes files in the venv and .git folder, even though they're supposed to be ignored by default, per the readme documentation.

I tried excluding and I still cannot get them excluded. I suspect this may be because I'm running this command on Windows 10. I'm using rsconnect-python==1.10.0

Here the command I'm using:

rsconnect write-manifest dash -o -e app:app .

The manifest is very large (5mb) because of these directories being included in the files section.

shapiromatron commented 1 year ago

I ended up creating a clean_manifest.py script that I can whenever generating an updated manifest, which works temporarily until items 2 and 3 are fixed below.

import json
from pathlib import Path

HERE = Path(__file__).parent

def main():
    """
    Clean the manifest does the following:

    1. Change locale setting to utf8
    2. Replace windows paths \\ with /
    3. Remove files in .git and venv

    An issue has been reported to fix the third item:
    https://github.com/rstudio/rsconnect-python/issues/285

    """
    fn = HERE / 'manifest.json'
    data = json.loads(fn.read_text())

    data['locale'] = 'en_US.utf8'
    new_files = {}
    for key, value in data['files'].items():
        if key.startswith('.git') or key.startswith('venv'):
            continue
        new_files[key.replace('\\', '/')] = value
    data['files'] = new_files

    fn.write_text(json.dumps(data, indent=2))

if __name__ == "__main__":
    main()
hcoohb commented 1 year ago

None of the folders that should be excluded by default actually are on windows. (.git, __pycache__, ...) And the glob pattern for file exclusion does not seem to work either on windows... This is very frustrating.

Any recommendation on how to deal with that?

shapiromatron commented 1 year ago

The "hotfix" is running the script above after generating the manifest, and before deployment. The correct way would probably be to not check paths in an OS-specific way (fwd vs backwards slash, using pathlib.Path for example) - see https://github.com/rstudio/rsconnect-python/blob/f595f69d0f943949bcc77f9f25710791ddadd454/rsconnect/bundle.py#L36-L49