search

fimad / scalpel

A high level web scraping library for Haskell.
Apache License 2.0
323 stars 43 forks source link

Please expose Internal modules :) #65

Closed typesanitizer closed 6 years ago

typesanitizer commented 6 years ago

Please consider exposing internal modules, even if they're not part of the versioning scheme. As a library user, it'd be much easier for me to jump through the docs via links and understand what's going on (and why some function is missing and whether I can implement it myself), if the Internal modules are made available.

fimad commented 6 years ago

Is this primarily an issue of documentation or are you looking to use internal functions in your code?

Is there a specific piece of functionality that you'd like but is missing? If so, I'd prefer to try and accommodate it by the official APIs rather than exposing an escape hatch to library internals.

typesanitizer commented 6 years ago

The issue I ran into was #21 but I see there are some other open issues as well, and I don't know when/if I might run into them :). I'll try to submit a PR over the weekend for 21. (I'll write a comment there discussing the implementation before making a PR). I think fixing that will help indirectly solve some other issues as well, partly if not fully.

I understand your sentiment of wanting to solve problems via the public API rather than a private API, and I wasn't used to it myself (outside of Haskell), but (a) it seems like the standard thing to do in Haskell-land (yes, this is a weird argument), and (b) it makes things more convenient in a pinch (I can use the Internal module without forking the library, if I really need it) without additional maintenance burden to you. If the library was a core library (e.g. containers), I'd understand that you also need to inform users about changes to private APIs, which would be an additional burden, but I'm hoping that won't be the case here. It would only work as a stop-gap solution between getting a task done and submitting a issue (PR) with a feature request (implementation).

I mentioned the docs because I thought that if the Internal docs were browsable online, then I wouldn't even need to clone the repo to see if what I was trying to do was possible by using Internal functions.

All said and done, I don't feel very strongly about this. If you still feel that the benefits don't seem to be worth it, I'm perfectly fine with closing this issue, no big deal :).

fimad commented 6 years ago

I see, thanks for the detailed explanation!

At the moment I'm leaning keeping the internal modules hidden (though I'm not necessarily opposed to exposing their documentation on hackage (if that's possible)). I've gone through several fairly dramatic rewrites of the internal code between different versions, I think anyone trying to depend on this would probably run into a lot of pain trying to upgrade between scalpel versions. Also, philosophically I think that if you can't do what you want with the public API, then its not good enough and it should be improved :)

I'll be curious to see what you think after writing a PR for #21. My intuition is that the majority of the things you'd want to do that aren't supported with the public API would require changing internal data structures anyway rather than just getting read access to them.

typesanitizer commented 6 years ago

Sure, no problem!