Soostone / hadron

Construct and run Hadoop MapReduce programs in Haskell
BSD 3-Clause "New" or "Revised" License
109 stars 11 forks source link

Fix WordCountLocal #5

Closed juanpaucar closed 4 years ago

juanpaucar commented 8 years ago

Hi, I was trying to get the examples to work and i had some troubles with this particular line. I don't know where does the function proto comes from, so I can't fix it for myself.

Thanks in advance.

juanpaucar commented 8 years ago

There also seem to exist errors there like

main = hadoopMain app (LocalRun def) RSReRun

where i get

Couldn't match expected type ‘[(Hadron.Controller.AppLabel,
                                        Controller ())]’
                    with actual type ‘Controller ()’
        In the first argument of ‘hadoopMain’, namely ‘app’
        In the expression: hadoopMain app (LocalRun def) RSReRun

i hacked something like

main = hadoopMain [(AppLabel "asas", app)] (LocalRun def) RSReRun

but of course i don't know if it is working yet, i don't know how to fix that proto

juanpaucar commented 8 years ago

There seems to be something to fix it in #4 with this code, but i'd still would like to know why did you change it, i'm still trying to figure out how to use the library but i'm stuck because i can't seem to fix the examples

MichaelXavier commented 8 years ago

@juanpaucar Hi Juan. Sorry for the lack of feedback on this. I've been going through our projects lately and updating the development/test side as necessary and I think hadron is due for some love here so I'll be looking into it. I'm going to get this project building with stack, set up a proper build matrix with travis and actually compile the example code as part of our test suite. After that, we can reassess and see if the example code is sufficient or if further documentation efforts are needed.

MichaelXavier commented 8 years ago

@juanpaucar Looking into this more, it looks like back in 2015 you had aims of becoming the maintainer of the lcs package seeing how it doesn't seem to build anymore. Indeed, the PR i'm reviewing now from #4 adds your fork as a dependency. I'm willing to keep that in for development but we can't release anywhere under these circumstances. Have you reviewed the taking over a package gudielines? If the author is still unresponsive, it sounds like it may be a good time for you to take over maintainership on hackage. Before this can release, we'll need an updated version on hackage (and preferably submitted to stackage). I'll then update the stack.yaml to point to the new version in the interim.

juanpaucar commented 8 years ago

Thanks a lot. I had already contacted with the author of "lcs" and we agreed that he would add me as a maintainer in hackage. I have contacted him once again already and i'd let you know ASAP what happens with that.

I really hope it gets better documentation, this package is so good that the first time I heard from it, it was on the ThoughtWorks' Radar Technology and not in a haskell related post. But, for example i'm still trying to figure up how to get it working with actual hadoop. Nevertheless, I must say that i'm not an advanced haskell programmer so probably that's the reason why reading the code it's not enough for me yet. I have seen how simple have been made multiple Map-Reduce steps yet I have no clue how to make it scale with multiple servers, should that be relied to hadoop itself?. Thanks for the attention and for sharing such a good package.

Stack's solver actually makes it dead easy to configure it with LTS, i'm currently having to work like this with the stack solver --update-config command. here's my stack.yaml file

flags:
  time-locale-compat:
    old-locale: false
packages:
- location:
    git: https://github.com/Soostone/katip.git
    commit: 666c4ce6dee6607f0f771eb284b0b0fd81ac0a9e
  subdirs:
  - katip
- location:
    git: https://github.com/juanpaucar/lcs.git
    commit: 40cc3bc32b5cd96d78f224850ec2d4428051b8b1
- '.'
system-ghc: false
extra-deps:
- BoundedChan-1.0.3.0
- Glob-0.7.5
- HUnit-1.3.1.1
- MonadRandom-0.4.2.2
- QuickCheck-2.8.2
- StateVar-1.1.0.3
- abstract-deque-0.3
- abstract-par-0.3.3
- adjunctions-4.3
- aeson-0.8.0.2
- ansi-terminal-0.6.2.3
- ansi-wl-pprint-0.6.7.3
- async-2.0.2
- attoparsec-0.12.1.6
- attoparsec-conduit-1.1.0
- auto-update-0.1.3
- base-orphans-0.5.0
- base16-bytestring-0.1.1.6
- base64-bytestring-1.0.0.1
- bifunctors-5.2
- blaze-builder-0.4.0.1
- blaze-builder-conduit-1.1.0
- byteable-0.1.1
- cassava-0.4.5.0
- cereal-0.4.1.1
- clock-0.6.0.1
- comonad-5
- conduit-1.2.6.1
- conduit-extra-1.1.9.2
- contravariant-1.4
- cpphs-1.19.3
- criterion-1.1.0.0
- cryptohash-0.11.6
- csv-conduit-0.6.6
- data-default-0.5.3
- data-default-class-0.0.1
- data-default-instances-base-0.0.1
- data-default-instances-containers-0.0.1
- data-default-instances-dlist-0.0.1
- data-default-instances-old-locale-0.0.1
- derive-2.5.23
- distributive-0.5.0.2
- dlist-0.7.1.2
- either-4.4.1
- erf-2.0.0.0
- errors-1.4.7
- exceptions-0.8.2.1
- extensible-exceptions-0.1.1.4
- free-4.12.4
- hashable-1.2.4.0
- haskell-src-exts-1.17.1
- hastache-0.6.1
- hostname-1.0
- hslogger-1.2.9
- ieee754-0.7.8
- kan-extensions-5.0.1
- lens-4.13.1
- lifted-base-0.2.3.6
- math-functions-0.1.5.2
- microlens-0.4.1.0
- microlens-th-0.3.0.0
- mmorph-1.0.5
- monad-control-1.0.0.4
- monad-par-0.3.4.7
- monad-par-extras-0.3.3
- mtl-2.2.1
- mwc-random-0.13.3.2
- network-2.6.2.1
- old-locale-1.0.0.7
- old-time-1.1.0.3
- operational-0.2.3.2
- optparse-applicative-0.12.1.0
- parallel-3.2.1.0
- parsec-3.1.9
- polyparse-1.11
- prelude-extras-0.4.0.3
- primitive-0.6.1.0
- profunctors-5.2
- quickcheck-instances-0.3.12
- random-1.1
- reflection-2.1.1.1
- regex-base-0.93.2
- regex-posix-0.95.2
- regex-tdfa-rc-1.1.8.3
- resourcet-1.1.3.3
- retry-0.7.1
- safe-0.3.9
- safecopy-0.8.6
- scientific-0.3.4.4
- semigroupoids-5.0.1
- semigroups-0.18.0.1
- split-0.2.3
- statistics-0.13.2.3
- stm-2.4.4.1
- streaming-commons-0.1.15
- string-conv-0.1
- stringsearch-0.3.6.6
- syb-0.6
- tagged-0.8.2
- tasty-0.11.0.2
- tasty-hunit-0.9.2
- tasty-quickcheck-0.8.4
- temporary-1.2.0.4
- test-framework-0.8.1.1
- test-framework-hunit-0.3.0.2
- test-framework-quickcheck2-0.3.0.3
- text-1.2.2.0
- tf-random-0.5
- time-locale-compat-0.1.1.1
- transformers-base-0.4.4
- transformers-compat-0.5.1.4
- unbounded-delays-0.1.0.9
- uniplate-1.6.12
- unix-time-0.3.6
- unordered-containers-0.2.6.0
- vector-0.11.0.0
- vector-algorithms-0.7.0.1
- vector-binary-instances-0.2.1.0
- vector-th-unbox-0.2.1.3
- void-0.7.1
- xml-1.3.14
- zlib-0.6.1.1
- zlib-conduit-1.1.0
resolver: ghc-7.10.2