Open masiulaniec opened 1 year ago
@masiulaniec Not sure if you are still looking at using reflow
, but for the original taskdb
problem, I think the following solution might work:
> reflow config -marshal > /tmp/reflow_config
> vim /tmp/reflow_config # and add the following line
taskdb: noptaskdb
> reflow run -config /tmp/reflow_config -local hello.rf
Perhaps @fialhopm might be able to confirm.
Apologies for the very late response.
Unfortunately, specifying noptaskdb
appears to not be sufficient to get hello.rf
to work in local mode.
If you're using the us-east-1
region, then the following should solve the resources exhausted
error:
> reflow config -marshal > /tmp/reflow_config
> vim /tmp/reflow_config # and remove the following instance types
- c6a.32xlarge
- c6a.48xlarge
- c6id.32xlarge
- g5.48xlarge
- i4i.32xlarge
- m6a.32xlarge
- m6a.48xlarge
- m6id.32xlarge
- r6a.32xlarge
- r6a.48xlarge
- r6i.32xlarge
- r6id.32xlarge
- trn1.32xlarge
- x2idn.32xlarge
- x2iedn.32xlarge
> reflow -config /tmp/reflow_config run -local hello.rf
This will not work for other regions.
We'll include fixes for both issues in the next release, which will hopefully go out within the next month.
I am trying to evaluate reflow but getting stopped in my tracks at Quick Start. I am simply trying to run the hello world example:
I initially assumed that local mode (Docker) would be the quickest. So I ran:
This smells like an internal error (dependency injection failure). It is surprising that TaskDB is a hard dependency when in
-local
mode. It contradicts the official documentation, which states that TaskDB is a soft dependency even for cluster mode.Having given up on Docker, I fell back on the official EC2 quickstart from the README. The
setup-ec2
/setup-s3-repository
/setup-dynamodb-assoc
trio worked fine. Unfortunely,reflow run
failed in a surprising way:The advertised
mem:0B
looks suspicious but I have not looked deeper than that.I tried a few older release builds but they all fail with the same error. If I go back far enough, I get a different error:
I was going to attempt some code fixups but here I encountered yet more trouble: the standard
go install
workflow does not work:My guess is that
go.mod
is not being kept in sync with the internal Bazel repo...I eventually managed to get it to build after a series of guesses around package upgrades and some local patching but by that point I lost any confidence that my local sandbox bears any resemblance to what upstream uses. Belatedly, I realized I maybe could have extracted an up-to-date
go.mod
from the buildinfo metadata embedded in the released binaries but I ran out of time dedicated to this experiment.Overall, a surprisingly poor experience for a project in its
1.x
life phase. It's a shame because the technology seems interesting.