zlatinb / muwire

MuWire file sharing client for I2P
GNU General Public License v3.0
191 stars 27 forks source link

Preserve directory structure when browsing? #65

Closed someguy2013 closed 2 years ago

someguy2013 commented 3 years ago

When searching for files, it would be nice if the directory structure were a part of the results. Or, especially if I choose to browse that user's files (assuming they allow browsing). There is an implied structure which can exist and would be meaningful to the person browsing compared to a flat unordered list of files.

I'd like to share /my-shared-files. Inside that, "readme.txt" and more subdirectories "shared-photos" and "shared-plans". I would definitely organized my shared matter that way. Why lose that organization on the browsing side of it?

I understand "collections" would serve this purpose to some extent (can there be collections within collections?). But, that seems like the hard way to get to the same place. If the directory is already a collection, why make the sharer stipulate this meta information again? And then distinguish between browsing files and collections. Wouldn't it be more intuitive to display the search results with their relative location among all the shared matter? Or, browse files and collections (files being part of directories, and directories within directories).

It just seems to me that there is inherent structure being lost.

[Sidenote: I wish MuWire didn't resolve symbolic links the way it does. It would be nice to share files without sharing their underlying directory. I.e., I organize /my-shared-files directory in a way that make sense for sharing, but not for actual storage on the disk. I don't think MuWire should ignore that, and sharing the underlying disk directory that I sym linked files within. (Again, this gets to my view that the /my-shared-files directory has the presentation/organization I would like searchers to see, and follow. If MuWire presented search results -- or at least browsing in this manner, then the symlink behavior would look more obviously incorrect.]

This is probably a feature request, not an issue. Thank you for your consideration.

zlatinb commented 3 years ago

Hi, thanks for your suggestions.

The main reason MuWire doesn't include directory information in regular search results as well as while browsing is exactly because it leaks information about the directory structure. When sharing a file, other than the initial hashing MW only knows the absolute path of the file; it doesn't know whether the file was shared as part of a directory or individually. That makes it impossible to know when to "stop" adding meta-info about the directory structure.

So to get something like this working MW needs to know when the user has finished sharing files, then analyze the tree structure to find the root. I'm not sure exactly to code such analysis; the approaches that come to mind are very error-prone and might by accident show the entire path to the file. I definitely do not want to show everything down to /home/username/...

With the collections approach the exposing of dir structure is opt-in, and that's why there exists a "preview" step when creating a new collection.

Regarding the symlinks, I see your point. I may make it optional in a future release, should be relatively easy to code.

zlatinb commented 3 years ago

Now that I thought about it some more it is possible to figure out a safe "root" to expose, but it would be a big change code-wise. I see the value in it, but it would be opt-in as well - the user will have to explicitly allow showing the directory structure in browse host.

I'll keep thinking to make sure all bases are covered and update the issue once there is something to test in git.

someguy2013 commented 3 years ago

I understand what you mean about directory names being a potential source of unintended information divulged by the sharer. But, that protecting the user is arbitrary by nature. File names could do the same thing. (Maybe only a hash value should be displayed. Said sarcastically, to make the extreme point.).

I've been thinking: what if you had a "safe" & "expert" mode? Safe mode would show shared files as a flat, no directory-structure view. Expert would let the user share their disk (or portion) more literally.

The client's Library tab already has this functionality with the choice between me (the sharer) viewing what I'm sharing as a "Tree" or "Table." I'm just thinking that the people on the discovery end might benefit from that same view toggle (if I allow it as "expert" sharer.).

It may not be worth the effort. I might be the only person who'd like to preserve/share the directory structure. To me, it seems like valuable meta info that adds to the shared files (being able to discover them through browsing, or figure out relationships that may exist. (Going back to my extreme example: even the filename is useful that way. You wouldn't prevent that from being seen.).

I understand what you mean about getting back to a safe root. I might share different branches (depths) of my /home/user/Documents directory. If someone found a file searching MuWire, then browsed me, that seemingly "disconnected" branching of directories would have to be rooted in a starting point for browsing. Like, maybe the all start with "/shared" (from that discoverer's point of view), and the disconnected branches grafted in there.

That's kind of what I was trying to accomplish when I tried using symbolic links. I created my own "shared" directory. Then, ln -s my shared files and directories inside that. Basically, doing my own grafting and having my own pre-made "view," leaving the shared things where they are. Creating my own portal so I can easily know/see what's shared. All would do is share that "shared" directory. Not share individual things here and there across my drive.

So, I hope that's just food for thought. It may not be important for it to work this way. It just seems like the discoverer could make more sense of what they're seeing if the sharer's structure were shown (and the sharer wanted it that way). But, I don't want to cause you do a lot of work for no real reason either.

Honestly: I think the idea of sharing one directory, and the sharer using symbolic links would make your program easier to develop/maintain. I, the sharer could put files in my shared directory (or use symbolic links if I want to preserve the way I keep things locally). I could place those files (or link to them) either flat or preserving the directory structures. I could share however I want that way. All MuWire would have to care about is: "which directory do you want to share from? (do whatever you want inside of it. Make it flat, or trees. Make it links to other places on your disk, or move those things into this directory."). To me, that sounds ideal and would put the sharer more in control of what they're sharing. "It all starts here. Be aware of what you're sharing. You have one place to see it all & verify it's what you intend. wysiwyg!" :)

Another nice thing about that way of treating it (aside from simplification for you) is that "collections" would be directories. MuWire's concept of a collection took me some time to understand what it means, how far it extends, what it can do, etc. I think the directory structure on the disk is more clearly a collection. It seems like that would be clearer, instead of trying to create a different "directory" within MuWire.

Food for thought. Thanks for your time developing and considering my thoughts.

zlatinb commented 3 years ago

Regarding the directory structure and a Tree/Table toggle in the Browse Host view, I've decided I'll definitely implement it. It just won't be for the next release which is due in less than 2 weeks because it's a significant change that involves network protocols as well.

Regarding the single shared folder, yes it would be much easier for me, but harder for the user. Symlinking is a pain on Windows. By the way, I2PSnark works exactly that way - there is one single shared directory and all torrents go there. Because of that creating a torrent in I2PSnark is way harder than sharing a file on MuWire.

More on symlinks - just today I found a problem when un-sharing a directory with symlinks that point to files in another directory. So I decided that I'm going to be implementing symlink support properly which means the user will be able to tell from the GUI that the file is a symlink. Again, not for the next release but hopefully for the one after.

Thanks for your feedback, I appreciate it!

zlatinb commented 2 years ago

@someguy2013 @searinox

I believe I have completed this feature. Please test latest from git if you can build from source, otherwise a binary is available here EDIT: binary removed as it produces a corrupt tree.

Since this is very new code, it's a good idea to back up your MuWire settings directory just in case.

Both the browsing node and the node being browsed must run this version in order for the directory structure to be displayed.

Searinox commented 2 years ago

Folder structuring doesn't match that of the shared machine. On one system I have several shared folders that continue similarly but are from different locations. Here is an example. If in Library it is:

Folder1\things\file1 Folder2\things\file2

The tree will sort everything as:

things\file1 things\file2

As if they were in the same location.

Next there are a number of files and folders from the internal folders just scattered about outside.

Finally, downloading a folder does not preserve the structure. If a folder contains multiple folders each with their files, all the files will be downloaded directly to the download folder with no preservation of structure.

zlatinb commented 2 years ago

The way the directory structure is determined is the following:

  1. If the file is in a folder that is not shared, it will appear "outside"
  2. If the file is in a folder that is shared, it will appear inside that folder
  3. If a folder is inside a folder that is shared, it will appear inside that folder.

To illustrate, let's say we have the following files on the hard drive:

c:\users\username\sharedFile1
c:\users\username\notSharedFolder\sharedFile2
c:\users\username\sharedFolder1\sharedFile3
c:\users\username\sharedFolder1\sharedFolder2\sharedFile4

For anonymity reasons, MuWire will not show c:\users\username or any of the folders which are not shared. The resulting tree will look like:

sharedFile1
sharedFile2
sharedFolder1\sharedFile3
sharedFolder1\sharedFolder2\sharedFile4

The specific case you describe of

Folder1\things\file1
Folder2\things\file2

makes sense if Folder1 and Folder2 are not shared. You can see which folders are shared when you look in Tools -> Advanced Sharing the Watched Directories tab.

Regarding downloading the structure like it's done for collections, I don't know if that's the right thing to do. But I'm willing to consider it.

Searinox commented 2 years ago

Example:

C:\programs\program1\bin....... with "program1" being the shared folder, and no negative tree so everything inside is shared

C:\programs\program2\bin...... with "program2" being the shared folder, and no negative tree so everything inside is shared

it will mash the two "bin" folder contents together and display one "bin" folder in the tree despite them having no relation.

zlatinb commented 2 years ago

Hmm I can't reproduce this. To share a folder you need to either drag-and-drop it on top of the LIbrary tab or select it for sharing via the "Share" button. Sharing everything inside a folder does not automatically share the folder itself.

We can try to debug this. Please add the following line to logging.properties:

com.muwire.core.files.PersisterFolderService.level=FINE

When you restart MuWire you will see many lines like:

[timestamp] FINE ... loaded file c:\full\path\to\file
[timestamp] FINE ... loaded shared parent from json parts\of\the\path

It should really be one line for each shared file.

I may need to add some more logging to see why it's not picking up the program1 and program2 folders. Please confirm that the loading shared parent from json loads only the bin part of the path.

Searinox commented 2 years ago

Just to be clear, this is an issue with the tree listed when Browsing a host not the Library listing. In the Library these will appear as:

programs program1\bin.....

program2\bin.....

and will be distinct.

It is only in the Browse tree view that I see:

bin\files from program1 and program2's bin

zlatinb commented 2 years ago

Please check if c:\programs\program1 and c:\programs\program2 appear in the list of "watched" directories in Tools -> Advanced Sharing. Those two folders must appear with exactly those names; disregard the \bin\ sub-folders that will also appear in the list.

Searinox commented 2 years ago

The folders program1 and program2 were dragged and dropped in the Library area. The folders themselves and every single file inside them have a checkmark on the watch box. Their full paths are listed. The \bin subfolders also appear. The names are correct.

In the Library tree there is "programs" which branches into "program1" and "program2" and the whole layout is correct.

In the Tree view there is no "program1" or "program2". There is only "bin". Then inside "bin" are all the files from "program1\bin" and "program2\bin" mashed together.

Let me help you with a more descriptive way to reproduce this:

-In a drive, let's say C:, create a "programs" folder. -In "programs" create two folders "program1" and "program2". -Inside each of the two folders create a "bin" folder. Do not add any other folders. -Within "program1\bin" and "program2\bin" respectively, put a few distinct files. Do not add any further folders down the path, only those files. Make sure the files in each folder do not match. Example: "file1.txt" and "file2.txt" in "program1\bin" and "file3.txt", "file4.txt" in "program2\bin". -Drag the "programs" folder into the Library and wait for it to be shared. -Using another instance of MuWire, use Browse to retrieve the sharing instance's folder.

In the Tree view, you should see a single "bin" folder as the root, which claims to have file1.txt, file2.txt, file3.txt and file4.txt inside it.

zlatinb commented 2 years ago

I followed the exact steps you describe on Windows 10 and cannot reproduce the problem. The tree looks correct:

image
zlatinb commented 2 years ago

I took a deeper look at the code and I think I see one possible way this could have happened - were program1 and program2 shared before you updated to the custom MuWire version?

I think the problem happens during the upgrade and the way to reproduce this is to have many shared files and many watched directories before trying the new version.

Did you make a backup of your MuWire settings directory?

Searinox commented 2 years ago

Yes they were indeed shared before updating. I will unshare and re-share now and try again.

zlatinb commented 2 years ago

I reproduced it - a problem with the upgrade. I will try to come up with a fix, this one is a little tricky.

zlatinb commented 2 years ago

Fixed in the last commit. I've tested this by converting 70k files and 9k directories and the final tree was correct. New binary available at https://muwire.com/downloads/MuWire-0.8.9-GitHub65-2.zip . I'm going to remove the first binary as it results in corrupted tree.

Searinox commented 2 years ago

I have taken the GitHub65-2 package, replaced it on both instances, then on one of the instances I unshared everything then re-shared the folders. Once all the files were done sharing, I shut down the sharing instance and re-opened it, and waited for it to reload everything.

On the searching instance, where I also put the new package, I selected the sharing instance and Browse. The folder tree unfortunately, still suffered the exact same issue. Everything was mashed together into one "bin" folder and their parent "program1" and "program2" folders weren't even in the tree. They continue to be listed with correct structure in the sharing instance's Library though. Though that was never an issue.

Searinox commented 2 years ago

In advanced sharing -> watched folders

c:\programs\program1\bin c:\programs\program2\bin

were showing with checkmarks for Auto sync.

I think I may have told you something wrong. Instead of sharing the two "program1" and "program2" folders, share the two individual "bin" folders instead.

zlatinb commented 2 years ago

Instead of sharing the two "program1" and "program2" folders, share the two individual "bin" folders instead.

That makes all the difference. Only watched folders are included in the tree, all folder information that is not watched gets lost. So when generating the tree MuWire hides the c:\programs\programX part. While the sharing node knows they are different bin folders, the browsing node doesn't know that so it combines them into one folder.

I agree it's not great but there's no way to fix it without showing more information than the sharing user is willing to give.

someguy2013 commented 2 years ago

the browsing node doesn't know that so it combines them into one folder."

I'm sorry I haven't participated in testing. I'm impressed with the priority this enhancement received. But, I wonder about that bolded bit I quoted. I'm sure I can live with the described behavior. But, blending duplicately-named folders (at the root) seems like it might be confusing or unintuitive. Why not have a faux "root" in such cases, like "../bin" and "../bin." Then the ".." would signify "part of a different unshared structure." The browser is just getting a view of what's shared. If there's two directories of the same name, maintaining their distinction seems better than folding them into one?

I agree that this is something the user could work around with more thought about how they share things. But, it seems like folding them into one "bin" (if I understood correctly?) would not lead them to that understanding. Whereas, I think two visible "../bin" "../bin" would cause an "aha" moment -- that the shared structure isn't unique enough (or is, and that two buckets is better than one larger one?).

Just my thoughts. Thank you for all you do.

zlatinb commented 2 years ago

Hi @someguy2013

The only way the faux root can be introduced is quite complicated - obfuscate the "hidden" part of the path and send that for every file. Obfuscation can be done by salting and hashing the hidden part with the same salt for all files. Then on the browsing node compare the hidden part when building the tree.

I don't really like this approach although it would work. The current behavior can be seen as a feature (I know, not a bug a feature lol) because it allows the user to build "virtual" folders from multiple physical folders.

Searinox commented 2 years ago

What if in the shared list MuWire assigns a unique ID to each unique root(the first bin has 0, the 2nd bin has 1 etc.) that it doesn't show to the user, but sends it when Browse is requested. Therefore the browsing instance would be receiving "0\bin...", "1\bin...." and we agree by convention that when a path is received, the first identifier represents a unique root. We don't show it to the user but we still group everything in the list as if they had unique roots. That way we will see multiple "bin" folders correctly. And when a file is selected for download, the sharing instance gets the full path(requesting download of "0\bin\file1.txt") and it knows which root to fetch it from.

Are request versions supported by MuWire? You could up the version counter for this. If someone makes a request using the old version, they will get the mashed up folders, missing the ID. If someone has a node that supports the new format, it will make a Browse/Download request with the new version and get the properly formatted data. That way we don't break compatibility.

zlatinb commented 2 years ago

Requesting files for download is done by the hash of the file, not by the path or name, so no changes are necessary there.

Assigning unique id to each unique root is much harder than it looks because it requires keeping track of all unique roots which change when user unshare files and so on. If we want to send unique roots to the browsing node it would have to be cryptographically generated (i.e. salt + hash).

But I'm still not convinced this is a real problem. How does having two bin folders instead of one in the result tree help you in any way if you can't see which program folders they belong to? Another counter-argument is having for example C:\CatPictures and D:\CatPictures. In this case it is actually better experience for the browsing user to see a single CatPictures folder.

I'd rather leave things as is and shift focus to making downloading a folder preserve it's structure.

Searinox commented 2 years ago

The folder merging came up as the current behavior of the application was discussed in the context of this issue. The issue remains directory preservation.

I have gone through a few attempts come up with an explanation that does not require me discussing the specific case that is affecting me. I don't think I'll be able to make my case as it stands without mentioning more.

The Library layout kept the "program1\bin" and "program2\bin" - without disclosing the original root - indicating what the original folder was and confusion would have been prevented if it was displaying the same way on the browsing user's side. This prevented the ambiguity.

I did not want any other stuff from the "program1" and "program2" folders to get shared, only the bin folders. With the tools I have available, I may be able to reorganize this by sharing the "program1" and "program2" folders and using the negative tree to exclude everything that isn't "bin". My hope is that as other files are changed in "program1\" outside of "bin", they won't be automatically shared by MuWire. But I will work with what there is.

You should thus just focus only on the main part - allowing browse and download of a shared path with the preservation of its folder structure, though personally I hope you will at some other point revisit this specific case.

Searinox commented 2 years ago

On topic: having created shares that do not feature any same-named folders or weird stuff like that, I have created a rather varied share with many different sorts of items. On the other instance where I initiated the Browse I have noticed some files and folders still scattered outside the main root folders. I am using the latest WIP you made available in the discussions. I continue to not be able to find any references to UI errors in the logs.

My guess is, if you try it with a share of similar size and scope, you would see it happening too. That's the best I have as far as repro goes sadly.

someguy2013 commented 2 years ago

"obfuscate the "hidden" part of the path and send that for every file." I was thinking along the lines of two internal representations of the shared files (their structure). One representation for display, with the "../" prefix of where a shared branch ends as far as the viewer is concerned. Another representation for MuWire to reach an actual file.

I can see how "../" wouldn't be unique enough in the "downloads" folder. I liked the suggestion for "shared-path1/bin" and "shared-path2/bin". To me, that would be intuitive when the user shares in a way that causes such root collision in the viewer (and downloader's end).

I just think more than one person's going to have heartburn when only one bin folder appears in their library tab. I wouldn't intuitively look in it to see if maybe my other bins are in it. I would focus on "It's shared. Why isn't it?!?" I think you'd have to color (orange?) that amalgamation folder to note a "non-unique root" condition. Then you run into accessibility issues (color-blind users). Name it "bin-multiple." (But, then, if you can do that, why not break them out and number them.).

I believe I saw that your allowed for symbolic links to work differently. I'm pretty sure that's how I'm going to use MuWire. I like the concept of sharing one folder, and symbolically link to everything I share from within that. I like that abstraction in my own hands, on my machine, instead of letting MuWire do it for me. I still think that could be a good way for MuWire to do it. Just allow a shared folder, and let the user manage how they put their shared material in that folder (moved, or linked). You could even allow the user to create the links within MuWire's interface. (You mentioned something about Windows's links being difficult. Maybe if MuWire allowed the user to create them through its interface, that might be better?). It seems like that "one shared directory" model would simply MuWire, and reduce the risk of leaking paths. The "container" would be more external. MuWire wouldn't have to know as much. A cleaner division between what it needs to know for creating links, vs sharing them.

zlatinb commented 2 years ago

I did not want any other stuff from the "program1" and "program2" folders to get shared, only the bin folders My hope is that as other files are changed in "program1" outside of "bin", they won't be automatically shared by MuWire. But I will work with what there is.

Ok that is a valid use case. I will go ahead and implement the "fake root" proposal. Note that it will require re-sharing everything from scratch in order to generate the fake

On topic: having created shares that do not feature any same-named folders or weird stuff like that, I have created a rather varied share with many different sorts of items. On the other instance where I initiated the Browse I have noticed some files and folders still scattered outside the main root folders. I am using the latest WIP you made available in the discussions. I continue to not be able to find any references to UI errors in the logs.

I haven't been able to reproduce this with a good-sized share. While you still have the varied share on disk, can you try the following two experiments (unshare everything before each experiment):

  1. share everything in one go - that is select all the root folders you want to share and drag-and-drop them on the library tab in a single pass
  2. share all the root folders one by one, dragging-and-dropping a single root folder at a time

Please pay attention to which files end up outside the root folders; if it's the same files every time you repeat the experiment it is one type of problem, if they're different it's another type of problem.

I just think more than one person's going to have heartburn when only one bin folder appears in their library tab.

That's not going to happen, the Library tab displays the shares correctly, it's the Browse Host tab tree that is merging the two bin folders together.

I believe I saw that your allowed for symbolic links to work differently.

No I haven't touched that code yet, and haven't really decided what to do about symlinks yet.

Searinox commented 2 years ago

What I can tell you is that pathing alone isn't it. I've found it happening in files and paths that contain apostrophe but then I've found for instance, a path which had no special characters whatsoever, just A-Z letters, no whitespaces, no unicode characters, and a perfectly standard filename, path length was nothing special, and yet they too were orphaned. I am going to start unsharing them and then sharing them again sequentially. I don't see special attributes like readonly or hidden or anything, and they have no special NTFS permissions.

someguy2013 commented 2 years ago

"I've found it happening ... yet they too were orphaned."

Just a thought: Are any of your shared files/directories have symbolic links? I feel like I experienced what you're seeing when I tried to share one "share" directory, and created my shared structure within it using symbolic links. MuWire didn't treat that the way I expected. Instead of mirroring that stucture, it drilled down the symlink and created its structure from what the targets the symlink resolved to. So, the symlink was like an alias for MuWire's benefit (not mine, nor the person browsing my intended structure).

I hope I'm not misstating that. I've kind of forgotten now, a couple months later. :) There was something about how MuWire handled symlinks which made symlinks less useful (IMO), and it could have looked like what you're seeing?

Searinox commented 2 years ago

I am not using any manner of symlinks, they are standard, plain folders with files. They aren't even shared in Windows via its file sharing system. There are also not any shortcuts. No virtual or mounted drive folders either.

Searinox commented 2 years ago

I was able to get rid of the orphaned files and folders. What I did was unshare their root folders and re-shared them one at a time, waiting until hashing was complete before adding another folder.

Initially the folders were added in rapid succession, with me dragging and dropping all of them seconds apart. I used drag-and-drop now too except this time I always waited for the hashing to finish.

I should note that this bug happened in spite of me having selected a single thread for hashing, so nothing was ever hashed in parallel. Another note is that some of the folders came from different drives. I mention this because I proceeded to do the same every single time on another machine yet I never encountered orphaned files or folders.

So the solution does indeed appear to be waiting for hashing to finish before adding more. But why, given that I don't multithread the sharing process?

zlatinb commented 2 years ago

@Searinox that is very useful to know. There may be a bug somewhere in the hashing logic. Can I ask you to try the experiment one more time, this time adding the root folders quickly to see if the orphans appear again?

Hint: Un-sharing takes some time. The fastest way to unshare many thousands of files and folders is to shut down MuWire, delete the files and directories folders inside the MuWire settings directory and start MuWire again - it will come up right away with an empty library.

Searinox commented 2 years ago

Again something that happens only on one of the systems: re-sharing one of the folders resulted in a few of the files still making it outside. Nothing else had been shared. The process did take while a while though because the content takes up considerable space. So I was able to reproduce this even with a single folder share. Though sharing them sequentially definitely made this happen a lot less. They are not the same files as in previous runs, I think this indicates some kind of race condition.

zlatinb commented 2 years ago

Question: is the folder that produces the orphans on a shared network drive?

Please try to repeat with just that folder with the latest build: https://muwire.com/downloads/MuWire-0.8.9-GitHub65-3.zip

Searinox commented 2 years ago

No. The files are on a standard disk drive and none of them make use of NTFS symlinks, mounted/virtual folders, or any other manner of exotic layouts. There are also no special attributes like read-only or hidden, or non-standard NTFS permissions. They do however make use of NTFS compression.

Searinox commented 2 years ago

It appears there was a little confusion on my part. Even though unsharing some of the other folders removed their orphaned files, there is one folder whose orphaned files persist, even long after unsharing. In essence these files cannot be unshared anymore. If your previous build did indeed fix the issue, I would be unable to tell as things are now. I am going to have to delete the files tracking shares and start over with a whole fresh new set of shared folders.

In my first attempt, I will share in quick succession to see if your 65-3 version fixes the issue. If I can still reproduce it, I will then try sequential folder sharing.

Searinox commented 2 years ago

Update: After a very long session of sharing the folders with everything added sporadically in quick succession while sharing was already ongoing, the resulting directory structure - when browsed from a remote host - contained no orphaned folders and, to the best of my in-depth scouting, no missing files either.

My last observation would be that in tree structure - in both the local Library and remote Browse - there is no sorting and the files and folders are listed in a disorderly fashion. But this may well be another issue altogether. The tree finally works as it should.

zlatinb commented 2 years ago

That is very good to hear although I'm not sure I fully understand what went wrong in the first place. The tree structure appearing and loading without any ordering is normal, it would be a separate effort to make it organized in any fashion.

I'm going to leave this issue open as there are two outstanding items:

If in the meantime you discover anything please let me know.

zlatinb commented 2 years ago

I've fixed the duplicate bin folder in the commit above, build 65-4 EDIT: build removed as it doesn't obfuscate the root well.

You have to un-share and re-share everything, and both the browsing and sharing node need to run this build. (If only the sharing node runs the build you will see random strings as roots).

zlatinb commented 2 years ago

And in build 65-5 I implemented preserving the folder hierarchy when downloading from Tree view. Hierarchy is not preserved when downloading from Table view. Also, the newly created folders are not automatically shared, may do that next.

EDIT: build removed as it doesn't obfuscate the root well

zlatinb commented 2 years ago

Build 65-6 available with HMAC for the hidden root https://muwire.com/downloads/MuWire-0.8.9-GitHub65-6.zip

zlatinb commented 2 years ago

Build 65-7 is out - if sharing of downloaded files is enabled, the folder structures downloaded from Browse View will get shared correctly. I.e. if someone later browses the downloader they will see the same directory structure.

https://muwire.com/downloads/MuWire-0.8.9-GitHub65-7.zip

I can't think of any other aspects to this issue so if there are no issues with this latest build I'm hoping to finally close it :)

Searinox commented 2 years ago

Everything I tried appears to be working with regards to download, sharing, browsing, and preservation of folder structures. I was unable to find any issues. If there are any further bugs discovered, they will likely deserve their own issue. Everything set out to do in here is working. You can close this. This has resulted in a big improvement in MuWire's usability.

zlatinb commented 2 years ago

Great! Thanks for all your help.

someguy2013 commented 2 years ago

I know this issue/request is closed. I apologize for not participating more when it was active. I wanted to mention a bug I see in 0.8.11.

This directory structure:

Parent dir/
  parent_file.txt
  sub dir/
    sub_file.txt

when viewed in "table" mode, it shows

/path/Parent
/path/Parent dir/parent_file.txt
/path/Parent dir/sub
/path/Parent dir/sub dir/sub_file.txt

The "table mode" seems to parse the directory names by space? Not showing the full directory name when that's the only thing to show for that table row?

FWIW: I'm using symbolic links to create that structure in the shared folder. I assume symlinks are ok to use now?

Related point about the useability of browsing by directory structure:

It would be nice if I could search myself and see the results in the "searches" tab. It is an uneasy feeling to see my "library" tab's "table view" showing the long path before the folder I shared. I believe that's not visible to searchers. If the searcher were included in the search, then that would be confirmation of what it looks like to others.

Or, if the "library" tab's "table view" used elipsis (...) for that not-shared part of the path? The sharer could hover over it to see the full path as a pop up? (Or, use a gray background for the displayed part of the path not shared, and hovering over that could say "This part of path not shared nor visible in search results?")

Seems like this could be firmed up a little so it's more informative about what's exposed?

zlatinb commented 2 years ago

The "table mode" seems to parse the directory names by space

I don't see it happening with real folders; I'll try to reproduce with symlinks.

It would be nice if I could search myself and see the results in the "searches" tab.

Uncheck the Options->GUI->Exclude local files from results checkbox and you will see yourself in the search results.

Or, if the "library" tab's "table view" used elipsis (...) for that not-shared part of the path? The sharer could hover over it to see the full path as a pop up? (Or, use a gray background for the displayed part of the path not shared, and hovering over that could say "This part of path not shared nor visible in search results?")

I like the general idea. I think I'll color the invisible part of the path gray.

Thanks for the testing and report!

zlatinb commented 2 years ago

The "table mode" seems to parse the directory names by space

I don't see it happening with real folders; I'll try to reproduce with symlinks.

I figured it out - has to do with the length of the path, if it's too long it gets broken at a space. Try resizing the column?

someguy2013 commented 2 years ago

Thanks! (You may have told me about that "exclude local..." option before. It sounds familiar.). Some thoughts:

1. In Library>Tree view. I think greying (or shading the background of) the unshared/basepath folders would make a lot of sense.

2. In Library>Table view, I think it would make sense not to show the unshared/basepath. But, keep the hover text the same, so if someone hovers, they see the full path. Then, the "long path, broken at a space" condition wouldn't occur as much?

I think those two would be informative about what's shared, what's behind the share (the basepath). Tree view would show everything (but the base would be grey text or shaded background. The table view would have the basepath visible through mouseover.). I think switching views, that would be intuitive (together).

3. Search results come up in expanded tree view. I think that makes sense to show the hits. But, if I click on "browse files" that too defaults to a expanded folder presentation. IMO, that seems overloading (too much info).

When I suggested "preserving directory structure when browsing," I was thinking about letting the visitor drill down follow the inherent structure of the filesystem. I was thinking it would be collapsed by default, and the visitor would get that top-level view (lay of the land) and could choose how to drill down. Like a graphical FTP interface? (if all the subdirs are expanded, then any informative text files at the top-level directory are at the very bottom of the display.). Those top-level directory text files might be informative about the content of the subdirs, but pushed to the bottom by revealing all the subdir content?

4. When browsing in tree view, the results aren't sorted. If #3 above (default collapsed view) were the way it was done, maybe it would be feasible to sort the results as branches are expanded?

5. I think I ran into a condition where, in Library>Tree view, I unshared the directory (right click in tree view). Reshared it, and it had missing subdirs/content. Stop/start, unshare, stop, start, etc. Tried different things. I had to delete everything in the "files" and "directory" .config folders (I think I knew to do this because you told someone to do it once?). That fixed it.

So, I'm thinking it could be beneficial to delete the related config into when a person unshares?

I'm just thinking out loud.

someguy2013 commented 2 years ago

I previously said: "1. In Library>Tree view. I think graying (or shading the background of) the unshared/basepath folders would make a lot of sense."

I was thinking about that more: What if (in Library>Tree view) the unshared base folders were "containerized" by the background which sets them apart (the shaded background, or a dashed line?), and that grouping had a "hide/expand unshared base-path" toggle?

IMO, that would be the best of all worlds. (And, if the table view didn't show the base-path, but mouseover did, that would give reasonable availability to that underlying detail.). In the tree view, if "hidden," then the shared path structure could shift left? That would conserve some screen space (the unshared basepath wasn't contributing to the view).

That would be elegant, IMO. Probably complicated to do. But, I think it would be very intuitive, and allow the user to move the basepath out of the way (but still have access to see it).