search

colis-anr / colis-batch

A tool to run colis-language on packages and corpora of packages
0 stars 0 forks source link

fixes for colis interface #7

Closed benozol closed 4 years ago

benozol commented 4 years ago

Adapt to PR #124 in colis-language

Niols commented 4 years ago

This doesn't seem to work for me with PR colis-anr/colis-language#124, I get the following:

0 ✓ niols@shannon ~/git/colis/batch $ make
dune build @install
File "src/lib/model/maintscript.mli", line 53, characters 9-40:
53 |   states:Colis.SymbolicConstraints.state list ->
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: Unbound type constructor Colis.SymbolicConstraints.state
File "src/lib/model/serialisable.ml", line 13, characters 24-60:
13 | type colis_filesystem = Colis.SymbolicConstraints.filesystem =
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: Unbound type constructor Colis.SymbolicConstraints.filesystem
make: *** [Makefile:4 : build] Erreur 1

2 ✗ niols@shannon ~/git/colis/batch $ git rev-parse HEAD ../language/
350a27c4938aaa80bf0aad877f3f14baa96f4bf8
../language/
Niols commented 4 years ago

Did I do something wrong?

benozol commented 4 years ago

Sorry I missed a commit on colis-anr/colis-language#124 - could you give it another try?

Niols commented 4 years ago

Here goes: http://ginette.informatique.univ-paris-diderot.fr/~niols/colis-batch/2020-02-24_14-29-49_631e18e_clean-colis-interface/

I only looked really fast but this seems pretty similar to what was there previously?

benozol commented 4 years ago

Thanks!

Most stats under Scenarios look ok and the changes reflect PR Incomplete behaviour for utilities, where unknown utilities and incomplete utilities (and options) are now distinguished. But there are slightly more unexpected exceptions (624→642), and slightly more scripts not uncovered (19188 → 19364). Is this ok?

The stats under Script are identical. Is this expected?

Niols commented 4 years ago

Most stats under Scenarios look ok and the changes reflect PR Incomplete behaviour for utilities, where unknown utilities and incomplete utilities (and options) are now distinguished. But there are slightly more unexpected exceptions (624→642), and slightly more scripts not uncovered (19188 → 19364). Is this ok?

This is OK. However, I don't understand why there would be more problems coming from unsupported utilities? Or would that be unsupported utilities previously hidden (but not anymore) by an other one?

Also, why did we loose so many scripts ran completely? I don't see what would have changed.

The stats under Script are identical. Is this expected?

Yes, they are stats about the parsing and the conversion. They are all syntactic and should probably not have been touched by your changes, right?

benozol commented 4 years ago

  1. why there would be more problems coming from unsupported utilities?

Between 2020-02-19_16-14-40_64a40fd_master and 2020-02-24_14-29-49_631e18e_clean-colis-interface/ I see a decline from the 47919 unsupported utilities in master to 43600 unknown utilities (unknown unknown) but there are only 146 (new) incomplete utilities (known unknown). Overall it's really not clear how the errors shifted.

I noticed some remaining confusion between unknown and incomplete in fixed them in colis-language #125.

  1. The stats for scenarios from clean-colis-interface state:

    50913 (45%) could not be run at all, 49441 (44%) were ran completely, and 12974 (11%) were ran partially. Out of a total of 63888 problems:

but 50913+12974=63887. What's the 63888th problem? :) (same on master)

  1. Under Report > Utilities the scores of the utilities are quite different but I don't know what is scored.
Niols commented 4 years ago

  1. OK, this is me being stupid and reading it the other way around.

  2. Interesting. I probably miss a +1 somewhere. I will investigate!

  3. Indeed, this has changed. Batch relied on the list of registered utilities to compute the scores. Basically, the score aims at giving an idea of how many scripts you would gain by specifying the given utility:

    • a specified utility has a score of 0
    • for each script where it appears, a non-specified utility gains a score equal to 1 divided by the number of non-specified utilities in that script.

    It looks like the list of registered utilities that batch gets is now empty. This messes with the scores: specified utilities get a score while they shouldn't, and the score of all the others gets lowered quite a lot.

benozol commented 4 years ago

@Niols could you please re-run the batch using colis-language#125? That PR contains a fix for the use of previously "unsupported" behaviour of symbolic utilities, which should reduce the differences in the stats between colis-language master and the functorization.

Niols commented 4 years ago

Why the force push though?

benozol commented 4 years ago

I rebased this PR on master and resolved the conflicts

Niols commented 4 years ago

The report will appear here before 10:30 Paris time.

Sorry for the delay, I got lost in the whole force push thing.

benozol commented 4 years ago

Thanks! This looks largely fine, but it surfaced an issue with the interpretation of dpkg, as being discussed in https://github.com/colis-anr/colis-language/pull/125.

Niols commented 4 years ago

The problem of registered utilities and score is still here though.

benozol commented 4 years ago

That should now be resolved in https://github.com/colis-anr/colis-language/pull/125 with https://github.com/colis-anr/colis-language/pull/125/commits/acb30bf7174885244904768da979b8b52eae834e

Niols commented 4 years ago

Indeed!