snivilised / pixa

🧙 Directory tree based bulk image processor (Also serves as a working example of how to use extendio, cobrass and arcadia)
MIT License
0 stars 0 forks source link

destination path not correct #172

Closed plastikfan closed 2 months ago

plastikfan commented 3 months ago

caused by the false assumption inside PathFInder.Result that the segment list is the same when the following condition isnt true:

f.transparentInput && f.Arity == 1

When the client is running a scheme where the arity is greater than one; ie the scheme contains more than 1 profile (which is actually the whole point of a scheme), then the current list of segments does not match the list of fields.

                return pfTemplates.evaluate(pfFieldValues{
                    "${{OUTPUT-ROOT}}":   to,
                    "${{SUPPLEMENT}}":    f.supplement(),
                    "${{ITEM-SUB-PATH}}": info.Item.Extension.SubPath,
                }, segments...)

the above field list does not match this field list:

[0]:"${{INPUT-DESTINATION}}"
[1]:"${{ITEM-SUB-PATH}}"
[2]:"${{DEJA-VU}}"
[3]:"${{SUPPLEMENT}}"
[4]:"${{TRASH-LABEL}}"

therefore, the resultant path is incorrect

When using blur-sf, profle, the actual incorrect result of the folder path is : "/blur-sf", but it actually should "tba". We need to check what the deficiency is in the unit tests, that allows for this incorrect condition not to be flagged.

For starters, the folder should be an an absolute path, but "/blur-sf", clearly isnt.

Actually, the output root (OUTPUT-ROOT) is being calculated as:

"/Users/plastikfan/dev/github/snivilised/pixa/test/data/research/scientist/nasa/exo/Backyard Worlds - Planet 9/sessions/scan-01"

so it should be joined with the current folder value of "/blur-sf"

Another thing wrong is that "/blur-sf" is just the scheme name. The profile should also be tacked on. The scheme/profile is the supplement, so the supplement field is nt being created properly (possibly because f.ExplicitProfile is not set).

The ExplicitProfile should not be used here. As the name suggests, its the profile that has been explicly set by the suer, but when running a scheme, the profile is not explicity set, it is derived from the scheme. The profile used should be takn from info *common.PathInfo, but running this test, shows that its not being set in here, so this needs to be fixed.

plastikfan commented 3 months ago

The problem with the tests not spotting the incorrect path stems from the fact that when running in dry-run mode, the actual output file is not created, a dummy is created and the path it creates is modified by the dry run decorator. But this falls short in those cases where a scheme is in play containing multiple profiles. The reason for this is, the dry run decorator aims the inform the user of approximate action, ie it fakes an output path that doesn't include the supplementary path (scheme/profile). But this falls short of correctness and therfore verification. We need to re-think the purpose of the dry run decorator and also allow for registering a callback that allows a test to see the paths being created by the path finder, so that they can be asserted, rather than asserting existence in the virtual file system.

plastikfan commented 3 months ago

code to create a profiles instance inline:

        profiles := cfg.MsProfilesConfig{
            Profiles: cfg.ProfilesConfigMap{
                "blur": clif.ChangedFlagsMap{
                    "strip":         "true",
                    "interlace":     "plane",
                    "gaussian-blur": "0.25",
                },
                "sf": clif.ChangedFlagsMap{
                    "strip":           "true",
                    "gaussian-blur":   "0.25",
                    "adaptive-resize": "60",
                },
                "adaptive": clif.ChangedFlagsMap{
                    "strip":           "true",
                    "interlace":       "plane",
                    "gaussian-blur":   "0.25",
                    "adaptive-resize": "60",
                },
            },
        }
plastikfan commented 3 months ago

there still seems to be a problem when a scheme is run. the path-fibder tests look correct, so the way the shrink command is interacting with the path-finder is still not correct, so it must be made to interact in the same manner as the tests.

eg a test for scheme:

        Entry(nil, &pfTE{
            given:  "🌀 TRANSFER: transparent/scheme/non-cuddled (🎯 @TID-CORE-5_TR-SC-NC_BLUR_T)",
            should: "redirect input to supplemented folder // filename not modified",
            reasons: reasons{
                folder: "transparency, result should take place of input in same folder",
                file:   "file should be moved out of the way and not cuddled",
            },
            scheme:  "blur-sf",
            profile: "blur",
            supplements: supplements{
                folder: filepath.Join("$TRASH$", "blur-sf", "blur"),
            },
            actionTransfer: true,
            assert: func(folder, file string, pi *common.PathInfo, statics *common.StaticInfo, entry *pfTE) {
                Expect(folder).To(Equal(filepath.Join(pi.Origin, entry.supplements.folder)), because(entry.reasons.folder))
                Expect(file).To(Equal(pi.Item.Extension.Name), because(entry.reasons.file))
            },
        }),

the expectation for the folder includes the profile with the scheme, but the scheme command indicates different behaviour:

pixa shrink ~/dev/test/pics --scheme "blur-sf" --files "wonky*" --dry-run

    - ⣻  status(🧙 dummy): 🔊 navigation completed ok (files: 3, folders: 0) 💝 [started: 'Mon, 19 Feb 2024 10:04:45 GMT', elapsed: '9.003415839s']
        -->        info: action: '⛔ dry with  single CPU', scheme: 'blur-sf', profile: 'blur'
        -->      source: 🦁 wonky.small-avatar.jpg
        -->        from: /Users/plastikfan/dev/test/pics/$TRASH$/blur-sf/
        --> destination: wonky.small-avatar.jpg
        -->          to: /Users/plastikfan/dev/test/pics/
        -->       error: 💥 could not create file at path: '/Users/plastikfan/dev/test/pics/wonky.small-avatar.jpg': file already exists
        -->    progress: 3 of 3

which shows the folder not including the profle component (in this case blur: /Users/plastikfan/dev/test/pics/$TRASH$/blur-sf/, but should include the profile: /Users/plastikfan/dev/test/pics/$TRASH$/blur-sf/blur/)

This issue is probably cauing the error: '💥 could not create file at path: '/Users/plastikfan/dev/test/pics/wonky.small-avatar.jpg': file already exists'.

In the test, the PathInfo object contains the scheme and profile properties populated, but OnNewShrinkItem creates the PathInfo using the inputs. The problem with this is the profile is not explicitly set by the user, so profile won't be set. The profile still needs to be set, but it is in the controller step, but something is still missing.

The problem is with Setup, which is invoked by the controller outside the context of a a profile, before it runs all the steps in the sequence which each all do have the profile. The question is should Setup be invoked by the Step instead of the controller. But the input can only be moved once, so it should only require scheme segment, not the profile, so the error must be coming from the Result path returned, as that doe have to have the profile segment.

Scheme is not compatible with TransarentInput and the reason is, there is no way the for multiple profiles to take the place of the input, which one replaces it? We should hardcode this restriction, ie, we set TransparentInput to false, when a scheme is active.

With this fix, during a dry run, now we see:

error: 💥 open /Users/plastikfan/dev/test/pics/blur-sf/blur/wonky.small-avatar.jpg: no such file or directory

This is actually ok, since during a dry run the input file is not transfered to this location, so the error should not be reported for dry run.

plastikfan commented 2 months ago

still not correct.

pixa shrink ~/dev/test/pics --profile "blur" --files "wonky*" --dry-run

    - ⣽  status(🧙 dummy): 🔊 navigation completed ok (files: 3, folders: 0) 💝 [started: 'Tue, 20 Feb 2024 07:24:00 GMT', elapsed: '9.005559761s']
        -->        info: action: '⛔ dry with single CPU', scheme: '[NONE]', profile: 'blur'
        -->      source: 🍰 wonky.small-avatar.jpg
        -->        from: /Users/plastikfan/dev/test/pics/$TRASH$/ADHOC/
        --> destination: wonky.small-avatar.jpg
        -->          to: /Users/plastikfan/dev/test/pics/
        -->       error: 💥 could not create file at path: '/Users/plastikfan/dev/test/pics/wonky.small-avatar.jpg': file already exists
        -->    progress: 3 of 3

shows that when profile is specified, the input is transfered to an 'ADHOC' supplemeted path, when it should just be the profile.

We need to fix the tests in pixa_test, to make sure they are in line with the path finder tests, to catch these types of errors.