facebook / buck

A fast build system that encourages the creation of small, reusable modules over a variety of platforms and languages.
https://buck.build
Apache License 2.0
8.56k stars 1.16k forks source link

Using a `python_library` build rule as a dependency to another build rule does not behave consistently #2618

Open swarren12 opened 3 years ago

swarren12 commented 3 years ago

The behaviour of python_library when used as a dependency to another build rule is currently inconsistent (or at least, the documentation doesn't make it clear what should and should not work).

The documentation provides an example of using a python_library as a dependency to a python_binary rule, implying that there is some output of a python_library that can be referenced by other targets:

python_binary(
  name = 'tailer',
  main_module = 'tailer',
  deps = [
    ':tailerutils',
  ],
)

python_library(
  name = 'tailerutils',
  # The main module, tailer.py, is specified here.
  # (Separated out from the glob pattern for clarity.)
  srcs = glob(['tailer.py', '*.py']),
)

Building the example works fine, although you can see that the //:tailerutils target does indeed produce no output:

$ buck build --show-outputs //:tailer
//:tailer buck-out/gen/tailer.pex
$ buck build --show-outputs //:tailerutils
//:tailerutils
$ ./buck-out/gen/tailer.pex          
hello world

However, trying to include //:tailerutils in other targets (using a trivial genrule as an example) doesn't seem to work, e.g.:

genrule(
  name = 'lstailer',
  cmd = 'tree * > $OUT',
  srcs = [
    ':tailerutils',
  ],
  out = 'lstailer.txt'
)

Attempting to build this target gives the following error:

$ buck build --show-outputs //:lstailer
No known output for: //:tailerutils
    When amending unnamedSources.
    When amending srcs.
    When amending buildableForRuleKey.
    When building rule //:lstailer.

In contrast, depending on the //:tailer target works as expected (which makes sense given that this does appear to have a well-defined output):

genrule(
  name = 'lstailer',
  cmd = 'tree * > $OUT',
  srcs = [
    ':tailer',
  ],
  out = 'lstailer.txt'
)
$ buck build --show-outputs //:lstailer 
//:lstailer buck-out/gen/lstailer/lstailer.txt
$ cat buck-out/gen/lstailer/lstailer.txt
buck-out
└── gen
    └── tailer.pex -> /home/example/buck-out/gen/tailer.pex

1 directory, 1 file

I would expect that, given the source files of the python_library target are made available to the python_binary target, this behaviour should be consistent across all build rules?

I wondered if there was a workaround to this by using some kind of string expansion macro that could be used in the genrule, e.g. srcs = [ '$(query_targets "inputs(//:tailerutils)"), but it appears that this also isn't made available.

(example: https://github.com/swarren12/buck-python-library-example/).