hledger-web on sandstorm is not satisfying [$100]

[simonmichael]

Negative feedback is accumulating at https://apps.sandstorm.io/app/8x12h6p0x0nrzk73hfq6zh2jxtgyzzcty7qsatkg7jfg2mzw5n90 , need better Sandstorm integration to fix this.

[ndarilek]

A thought I had: Is there any current Git integration? Can the web interface make a commit whenever a transaction is added or modified, or does it already?

One feature I've played with for some more recent Sandstorm apps I've developed is backing them with a Git repo. If the web interface could detect or be told that the ledger is in a repository, and if I can expose that repo via Git, then the ledger could be updated both via the web interface and by the repository. Not sure how to handle incoming pushes while the web interface is open, so the idea obviously needs some additional thought.

[simonmichael]

No, we don't have that feature yet. It would be nice. filestore is a haskell library that can talk to git and other VCS's.

[AaronM04]

FWIW, my hledger-web on sandstorm.io + git workflow is as follows:

• create all transactions in the web interface (there's no way to import a ledger file, and no way to update it except by adding transactions with the web interface).

• periodically use the Download Backup button in the middle of the top bar, and then run a shell script I created that moves the .zip file from my Downloads directory, extracts it, commits the ledger file, and finally pushes it to GitLab.

[simonmichael]

there's no way to import a ledger file

Actually there is; just reverse the backup I think. It was described recently on the hledger mail list.

[AaronM04]

@simonmichael Cool, I didn't know about that. 💯 What's the link to this thread?

[simonmichael]

https://groups.google.com/d/msg/hledger/5BTfowLOkpE/TqVuRjaoDQAJ

[simonmichael]

The sandstorm reviews situation is getting pretty serious. I would almost prefer we were removed from Sandstorm until ready rather than keep accumulating bad first impressions and bad reviews. @ndarilek is there a way to mark an app as experimental/work in progress/not ready for review ?

[simonmichael]

Also, is there any way to contact reviewers, to understand their issues better ? Eg "Could not find a way to enter a single transaction".

[simonmichael]

I will work on some new text for the app description, to try and manage expectations.

PS, also version 1.1 is out, though it won't affect the sandstorm app much.

[ndarilek]

I'm not aware of either a way to mark apps as experimental, or to contact reviewers. Sorry. I can ask to have the app pulled if you like, though I don't know enough Haskell to do any work on it. I was just looking for a quick and secure way to run the web interface and thought I'd make a package.

[simonmichael]

It's appreciated @ndarilek. Let's see what we can do with the description, which I'll work on later. Possibly we should open new issues or start a checklist here or a new Project to track the sandstorm issues reported.

[simonmichael]

Getting back to this. I've updated my app review to include a link to this issue and a call for help. @ndarilek, I think only you can update the app description, right ? Can you update to:

hledger (http://hledger.org) is a dependable, precise, cross-platform
program for tracking money, time, or any other commodity, using
double-entry accounting and a simple plain text file format.  It is a
haskell reimplementation of Ledger, provides command-line, curses and
web interfaces, and aims to be a robust, practical tool for daily use. 

Sandstorm users, thanks for your feedback. hledger's web UI is a simple one 
to fit current development resources, and I'm sorry that Sandstorm integration is lacking. 
Your help is needed. We have an open issue for this where you can add a bounty or 
contribute patches ( https://github.com/simonmichael/hledger/issues/425 ). 
We also welcome bug reports with more detail ( http://bugs.hledger.org/new ), 
general hledger-web improvements, backers and sponsors ( http://hledger.org ).

I expect this will need ongoing updates, so feel free to add me as a co-editor if that's possible.

[simonmichael]

Testing help wanted: please try to find out why three reviewers have said "Could not find a way to enter a single transaction.". There's supposed to be a visible "Add a transaction" link, and the a key mentioned in the help accessible from the ? button, as at http://demo.hledger.org.

[agarrharr]

I think the reason they couldn't add a single transaction is because you have to understand double-entry accounting. Currently, if you enter a new transaction that doesn't balance, then it just clears out the input fields. It would be better if it left your input fields filled in and gave you an error message, saying that it doesn't balance, and perhaps with a link to a helpful explanation.

[simonmichael]

Ah! Thanks, that makes sense.

[ocdtrekkie]

@simonmichael I found this issue poking around Sandstorm's reviews. I can definitely say I was confused with what this accounting did, and I think after some fiddling/experimenting I managed to make it do things, but yeah, the "double-entry accounting" bit is completely lost on me.

Generally I'd expect an error message if something wasn't right, not for it to dump the data that I entered, as @agarrharr said.

As a note, if someone is interested in working on the Sandstorm package in the future, I can help with that, I've got a strong desire to help move the app situation on Sandstorm forward as a whole.

I don't feel that the review situation is as tragic as it appears: A lot of Sandstorm packages are currently out of date, and that's a relatively well known problem amongst the Sandstorm community. (That being said, I'd love to someday see the way reviews are presented improved, such as displaying which version a given review was made from, and one could argue new reviews should be weighted more heavily than old reviews.)

[simonmichael]

@ocdtrekkie any help would be welcome! Nobody is working on the sandstorm support currently.

[ocdtrekkie]

As the case is, I now have the publishing key for pushing updates to it. I am not much of a coder though, and have never touched Haskell in my life. Is the state of HLedger Web compared to where it was as of the previous Sandstorm package, that there'd be significant value in just getting a refreshed/current Sandstorm build out there? I am reasonably good at version bumping things. :)

[simonmichael]

That would be great, if you can assist or take over from @ndarilek. We are happy to help with haskell know-how in the #hledger IRC channel.

https://apps.sandstorm.io/app/8x12h6p0x0nrzk73hfq6zh2jxtgyzzcty7qsatkg7jfg2mzw5n90 looks like it's based on 1.1, which is about 1.5y behind the current 1.9.2. There has been no major hledger-web work for a while, but http://hackage.haskell.org/package/hledger-web/changelog shows some useful fixes. There may be other useful changes mentioned in http://hackage.haskell.org/package/hledger-lib/changelog .

[simonmichael]

Tips:

• http://hledger.org/download.html#c
• stack install hledger-web
• stack test hledger-web
• stack install ghcid; make ghcid-web

[ocdtrekkie]

So, the current Sandstorm version repo Nolan used is very simple, and everything is contained in the /.sandstorm directory. How would you feel about bringing it "in-house"? Nolan isn't actively maintaining this, so bringing it here might not be a bad idea, otherwise I can fork his repo.

https://github.com/ndarilek/hledger-sandstorm

I have bad Git fu (the GitHub Desktop app is my secret shame), but I wonder if it's possible to pull his commits in, preserving his authorship, and "just" removing the submodules bit, and adding the .gitignores together.

[simonmichael]

Merging .sandstorm in the main hledger repo sounds good if there's a good chance it'll continue to be maintained. Perhaps hledger-web/sandstorm ?

I'm not a git expert, but yes I bet you can do that.

[simonmichael]

PS hopefully we'll hear confirmation from @ndarilek in due course.

[ocdtrekkie]

I may experiment with this this weekend. I was reading http://voidcanvas.com/cherry-pick-format-patch/ just now. Presumably I can fork HLedger, make a branch, cherry-pick the 15 commits from Nolan's repo, make one commit of my own after that to remove the .gitmodules and adjust the .gitignore so it adds to your existing .gitignore instead of replacing it.

Then I'll need to update the metadata for the version, point the source repo back here directly, update the packaged by signature, and then I should, in theory, be able to package it at that point and submit a PR to merge that branch in here. As a note, I should also update the description as a part of this, based on the discussion above.

[ndarilek]

Yeah, no worries, and authorship preservation is a non-issue for me. Sorry I don't maintain this anymore--Sandstorm never took off in my organizations the way I hoped it would, and I need a bit more in the way of permissions/easy outbound network access than it currently allows. So, IOW, I'm just glad this app has a home, and don't mind losing authorship info if that's what it takes to continue with it.

[simonmichael]

It looks like you can almost just merge the hledger-sandstorm repo; but ideally we'd ignore the changes to anything but .sandstorm. Probably overkill, but if all else fails http://www.catb.org/esr/reposurgeon/reposurgeon.html could do the job.

[ocdtrekkie]

@simonmichael As I said, I think I can just drop the .gitmodules, and merge the .gitignore manually in a single commit at the end of my branch. You'll want the small addendum to .gitignore so anyone who works on the Sandstorm build doesn't accidentally commit files generated as part of the build process.

Helpfully, since I don't have commit rights to either yours or his repos, I can do a lot of bad experimenting, without doing anything destructive with my git commands. :)

[ocdtrekkie]

Another worthwhile thought: Once I get the package updated and upstreamed here, either I can sign the package and do releases, or if you are so inclined, I can transfer the publishing key to you, and you can do so directly. (When it's working, it should only be a couple of commands to push a release.)

Other people packaging and maintaining apps for Sandstorm is common, but some of our best-maintained apps are released straight from the original developer. It really depends on how much desire you have to support the platform or users using the platform. (Which probably has a close relationship with how often people ask you about the Sandstorm version.)

Additionally, if HLedger Web on Sandstorm can be made to be a good representation of what HLedger Web can do, you can actually use Sandstorm.io to provide a free demo of the app, with a link such as this: https://oasis.sandstorm.io/appdemo/8x12h6p0x0nrzk73hfq6zh2jxtgyzzcty7qsatkg7jfg2mzw5n90

[ocdtrekkie]

Pull request #789 makes no changes to fix/update the Sandstorm build yet, it just brings the existing commits in house. And it was a horror nightmare of looking for git commands on Stack Overflow.

[ocdtrekkie]

I updated some of the vagrant-spk files to more modern versions, but I hit this:

Downloaded nightly-2018-04-25 build plan.
AesonException "Error in $.packages.cassava.constraints.flags{'bytestring--lt-0_10_4']: Invalid flag name: \"bytestring--lt-0_10_4\""

I tried adding --resolver nightly to my stack install as I found in one issue, but that didn't get me anywhere so I took it back out. (This is in /.sandstorm/build.sh)

[simonmichael]

All good thoughts. I would love hledger-web to be great on Sandstorm, but I can't afford much time on that myself. I heard only one squeak about Sandstorm (see above) in the last year. The app is highlighted on our download page. If you would be able to act as hledger-web sandstorm packager, that would be a big help. That aeson issue looks like one that requires stack 1.6.x, I recommend always using the latest stack release (1.7.1) to minimize such breakages.

[ocdtrekkie]

Hmm, at https://github.com/ocdtrekkie/hledger/blob/upstream-sandstorm/.sandstorm/setup.sh it is adding a repo to install stack for Debian Jessie. If I change that to Stretch, do you think I'll get the newer version of stack?

[ocdtrekkie]

Current instructions have different installation suggestions, I'll go ahead and swap them out and see where I get. https://docs.haskellstack.org/en/stable/install_and_upgrade/

[ocdtrekkie]

Status update is that I made progress last night replacing basically the entirely of setup.sh with the curl | sh from the install instructions, hit a new issue, figured out the likely solution... and then fell asleep. So I'll probably find time to make additional progress later today. (Specifically, I think it ran out of RAM in the VM trying to install everything, so I need to adjust the Vagrantfile to be a bit more generous.)

[ocdtrekkie]

I kept getting RAM allocation errors and upping the amount of RAM I let the build tool use. Currently I'm at this, if you have any ideas?

Progress 133/185: haskell-src-exts-1.20.2, language-javascript-0.6.0.11, vector-                                                                                language-javascript-0.6.0.11: copy/register                    
vector-0.12.0.1: copy/register                                 
Progress 136/185                 

--  While building custom Setup.hs for package wizards-1.0.2 using:
      /home/vagrant/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_2.2.0.1_ghc-8.4.2 --builddir=.stack-work/dist/x86_64-linux/Cabal-2.2.0.1 build --ghc-options " -ddump-hi -ddump-to-file -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1
    Logs have been written to: /opt/app/.stack-work/logs/wizards-1.0.2.log

    Configuring wizards-1.0.2...
    Preprocessing library for wizards-1.0.2..
    Building library for wizards-1.0.2..

    on the commandline: warning:
        -XOverlappingInstances is deprecated: instead use per-instance pragmas OVERLAPPING/OVERLAPPABLE/OVERLAPS
    [1 of 5] Compiling System.Console.Wizard.Internal ( System/Console/Wizard/Internal.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/System/Console/Wizard/Internal.o )
    [2 of 5] Compiling System.Console.Wizard ( System/Console/Wizard.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/System/Console/Wizard.o )
    [3 of 5] Compiling System.Console.Wizard.Haskeline ( System/Console/Wizard/Haskeline.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/System/Console/Wizard/Haskeline.o )
    [4 of 5] Compiling System.Console.Wizard.BasicIO ( System/Console/Wizard/BasicIO.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/System/Console/Wizard/BasicIO.o )
    [5 of 5] Compiling System.Console.Wizard.Pure ( System/Console/Wizard/Pure.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/System/Console/Wizard/Pure.o )
    /usr/bin/ld.gold: error: cannot find -ltinfo
    collect2: error: ld returned 1 exit status
    `gcc' failed in phase `Linker'. (Exit code: 1)
Connection to 127.0.0.1 closed.
Command failed with a non-zero exit status (1).
jweisz@Inspiron-620:~/Projects/hledger$ 

[simonmichael]

@ocdtrekkie this is the issue mentioned at http://hledger.org/download.html#link-errors . stack doesn't know how to install C libraries. It would be very helpful if you can figure out the precise debian package which solves this (and any similar errors), and I'll update that list. For this one I would guess apt install libtinfo5.

[ocdtrekkie]

I tried installing libtinfo5, no dice. Tried libncurses5, which is what the linked article suggests for Debian installs, also no dice. (libncurses5 was already installed, so it says.)

[simonmichael]

Ah.. https://stackoverflow.com/questions/48855854/gcc-failed-in-phase-linker-cant-build-hint-package suggests it is libtinfo-dev that's needed.

[ocdtrekkie]

It builds! Though now I need to destroy the VM I've been playing in and recreate it from scratch to make sure just that install command will work. Then I just need to update the metadata and stuff.

[ocdtrekkie]

Do you have an example of an entry I can make which won't error out/cancel on me, so I can test things?

[ocdtrekkie]

Got it. One of the things that throws me is I tend to use 2018/06/04 (or 06/04/2018) as a date format, and hledger-web doesn't like that very much, it wants dashes. When I used the date picker, it was much easier to get a working submission. While some sort of date validation might be nice to correct the date format automatically, in the meantime, providing an example of how it should be entered on the form would probably help?

From the screenshot above, it doesn't look like the old Sandstorm version even had the date picker, so that would definitely explain some of the cause of failure to submit a transaction.

[ocdtrekkie]

Started updating the metadata. As a middle paragraph to the description you had requested above, I typed up:

In double-entry accounting, each transaction must be balanced: An equal amount added to one account must be subtracted from another. You may wish to consult hledger documentation or other tutorials on the web to learn how to use double-entry accounting.

I was trying to find a good first link to send people to to explain it, but I didn't find one that put it into a something simple enough to tell me "this is how you use it", and I am hoping this is correct enough a description to use?

[ocdtrekkie]

Improved metadata can now be seen at: https://apps.sandstorm.io/app/8x12h6p0x0nrzk73hfq6zh2jxtgyzzcty7qsatkg7jfg2mzw5n90?experimental=true (and the app can be tested there)

• Packaged by info and signature updated to mine
• Issues/source repos have been directed here
• License correctly identified as GPL3 instead of GPL2
• Updated description should alleviate confusion

If this is good to you, @simonmichael, I'll push one more commit to #789 with the updated metadata, and publish this update.

[simonmichael]

Got it. One of the things that throws me is I tend to use 2018/06/04 (or 06/04/2018) as a date format, and hledger-web doesn't like that very much, it wants dashes. When I used the date picker, it was much easier to get a working submission.

That looks worthy of a bug report. hledger-web is supposed to accept the same date formats valid in the journal (http://hledger.org/manual#simple-dates). From quick testing it looks like the date picker auto-converts to dashes if you paste, but if you manually type slashes it showed the add form again and added nothing to the journal.

[simonmichael]

I was trying to find a good first link to send people to to explain it, but I didn't find one that put it into a something simple enough to tell me "this is how you use it", and I am hoping this is correct enough a description to use?

Sure, it looks correct. There are a lot of intros on hledger.org and plaintextaccounting.org. What about things in the User Cookbook > Getting started. None of them are geared towards hledger-web data entry; you might have to write that. There's also the hledger-web manual.

[ocdtrekkie]

I am a little baffled that #430 is fixed in demo.hledger.org but not fixed in my new Sandstorm package which is built on master (as of the date when I made my branch). I am wondering if there is some part of the build process or Sandstorm that strips out href="#" for some reason.

[ocdtrekkie]

@simonmichael Arguably, #433 is still open with regards to the date bug, though it was significantly improved by the date picker being added.

[simonmichael]

Yes agreed, that is an annoying UX bug.

[ocdtrekkie]

Oh, nevermind, I didn't update my test grain properly! #430 is fixed on Sandstorm as well, that one should be closed.

[ocdtrekkie]

I am going to publish the new version because it is better than the existing with regards to the description and date picker in particular. It's objectively better than the existing release.