snivilised / extendio

🐋 extentions to Go standard io library
MIT License
1 stars 0 forks source link

cache isdir status on traverse item #413

Closed plastikfan closed 9 months ago

plastikfan commented 9 months ago

This is required because sometimes, file items can be transient in nature. eg, the journalling feature in pixa needs to be able to delete journal files during traversal. But this deletion is interfering with the navigator because it attempts to query the directory status of a file after it has been deleted.

To fix this, we should change IsDir on the traverse item to cache the directory flag so that the underlying isdir functionality never has to be queried.

plastikfan commented 9 months ago

this method was overlooked in addressing this issue:

func (a *navigationAgent) notify(params *agentNotifyParams) (SkipTraversal, error) {
    skip := SkipNoneTraversalEn

    if params.readErr != nil {
        if a.doInvoke.Get() {
            clone := params.current.clone()
            clone.Error = i18n.NewThirdPartyErr(params.readErr)

            // Second call, to report ReadDir error
            //
            if le := params.frame.proxy(clone, nil); le != nil {
                if errors.Is(params.readErr, fs.SkipAll) && (clone.Entry != nil && clone.Entry.IsDir()) {
                    params.readErr = nil
                }

                return SkipAllTraversalEn, i18n.NewThirdPartyErr(params.readErr)
            }
        } else {
            return SkipAllTraversalEn, i18n.NewThirdPartyErr(params.readErr)
        }
    }

    return skip, nil
}

Need to make sure there are no more calls to Entry.IsDir or Info.IsDir, they should always use the one on TraverseItem

Actually, this code looks like it was old code that should have been updated, long ago.

plastikfan commented 9 months ago

To distingusih betweed the is dir functionality on TraverseItem and FileInfo.IsDir/DirEntry.IsDir, rename TraverseItem version to IsDirectory.

Note: the sampling-while-iterator deals with fs.FileInfo instances directly, but for now this is ok and IsDir is called directly on it, because with its current design, doesnt have access to a TraverseItem instance to use instead.

plastikfan commented 9 months ago

Still failing

end to end REAL [It] should: tinkle the ivories
/Users/plastikfan/dev/github/snivilised/pixa/src/app/proxy/controller_test.go:584

  [FAILED] execution result non nil (lstat /Users/plastikfan/dev/test/pics/screen-shot-1.png.$journal.txt: no such file or directory)
  Expected
      <*fs.PathError | 0xc000592030>:
      lstat /Users/plastikfan/dev/test/pics/screen-shot-1.png.$journal.txt: no such file or directory
      {
          Op: "lstat",
          Path: "/Users/plastikfan/dev/test/pics/screen-shot-1.png.$journal.txt",
          Err: <syscall.Errno>0x2,
      }
  to be nil
  In [It] at: /Users/plastikfan/dev/github/snivilised/pixa/src/app/proxy/controller_test.go:610 @ 01/18/24 14:40:44.023
plastikfan commented 9 months ago

this could be a problem:

func (a *navigationAgent) traverse(params *agentTraverseParams) (*TraverseItem, error) {
    for _, entry := range params.entries {
        path := filepath.Join(params.parent.Path, entry.Name())
        info, e := entry.Info() // <---- PROBLEM!!!!

...

func (d *unixDirent) Info() (FileInfo, error) {
    if d.info != nil {
        return d.info, nil
    }
    return lstat(d.parent + "/" + d.name)
}

... and this is where the lstat error is coming from.

plastikfan commented 9 months ago

so if we see a PathError, then ignore it. Actually, it is better just to implement the exclusions issue. But as a test, in pixa, let us filter out journal files in the ReadDirectory hook, which we have already overridden to ignore .DS_Store files.

plastikfan commented 9 months ago

was resolved by modifying the ReadDirectory hook function to filter out journal files.