files mentioned in MANIFEST.in are not included in source tarballs anymore

[sebix]

Hi,

The current MANIFEST.in is:

https://github.com/MISP/PyMISP/blob/2a9c79a1e91a36d48ec992f972aa2878de547699/MANIFEST.in#L1-L10

However, out of these files, only the README and the data files are part of the source tarballs at pypi. Everything else is missing.

Maybe this change is related to the switch to poetry?

[Rafiot]

Hmmmm the files listed here are in the tarball here: https://pypi.org/project/pymisp/#files

[sebix]

Interesting. Not for me:

> wget -q -O - https://files.pythonhosted.org/packages/source/p/pymisp/pymisp-2.4.123.tar.gz | tar -tzf - | egrep '(docs|examples|tests|CHANGELOG|README)'
pymisp-2.4.123/README.md
pymisp-2.4.123/pymisp/data/misp-objects/README.md
pymisp-2.4.123/pymisp/data/misp-objects/docs/time-related-objects.ods
pymisp-2.4.123/pymisp/data/misp-objects/docs/time-related-objects.pdf

[Rafiot]

Ok, this one is missing: pymisp-2.4.123/README.md I mainly checked the object templates, which are required at runtime by the library.

The other ones should not have been in in the first place, especially the the content of pymisp-2.4.123/pymisp/data/misp-objects/docs/. The readme in misp-objects/ is also not needed or useful, afaik.

I'll re-add the the PyMISP readme for the next release, thank you for the report.

[sebix]

Ok, and what about the changelog, docs/, tests/ and (less important, but still documentation) examples/?

[Rafiot]

I'm never looking inside the packages for tests, examples, or documentation, but if you want to get them from there, sure, I can add them ;)

[sebix]

Then everything belonging to the version would be bundled in one place, the source tarball: the docs and the tests (which are necessary to validate the functionality of the code). IMO they should be part of the source.

It would definitely make my life easier (and likely also those of other packagers) :)

[Rafiot]

I can do that, but for the documentation for example, do you want the compiled version? It's going to be pretty big. And for the tests cases, how do you handle it in case you have big files used for the tests?

[sebix]

I can do that, but for the documentation for example, do you want the compiled version?

No, just the source, as in the ("source") repo itself.

And for the tests cases, how do you handle it in case you have big files used for the tests?

If they are part of the tests, they should be included IMHO.

[Rafiot]

I will add the source for the documentation, but the test cases files are over 40Mb. I can probably reduce it, but bundling PE executable in the package is something I'll not do.

Potentially, I can have a few different sets of testcases, and if testcases require files that aren't present, I skip them.

[sebix]

I will add the source for the documentation

Thanks

but the test cases files are over 40Mb. I can probably reduce it, but bundling PE executable in the package is something I'll not do.

Are these the (outhoused) viper testfiles? I think they could be skipped, yes.

[Rafiot]

but the test cases files are over 40Mb. I can probably reduce it, but bundling PE executable in the package is something I'll not do.

Are these the (outhoused) viper testfiles? I think they could be skipped, yes.

Yes. I just need to add some exceptions in the test suite so it passes without the submodule.

[sebix]

but the test cases files are over 40Mb. I can probably reduce it, but bundling PE executable in the package is something I'll not do.

Are these the (outhoused) viper testfiles? I think they could be skipped, yes.

Yes. I just need to add some exceptions in the test suite so it passes without the submodule.

I can also work on that (or at least try to).