use an external command as a parser

[masatake]

Like --regex-= option introduce --cmd-= option.

Some people wrote their own tag generator for a language they are interested in. e.g. https://github.com/lukaszkorecki/CoffeeTags Good news is that the format of tags file generated by those generators is compatible with format defined at Exuberant Ctags project.

fishman ctags can be onestop unified front end for those generators. If we use pipe to communicate with external generators, readtags is good place to start.

[masatake]

I have inspected Coffee Tags. Some small modificatoin is needed to use it as backend parser. https://github.com/lukaszkorecki/CoffeeTags/pull/45

If my patches are accepted, I will hack fishman-ctags side.

[masatake]

One more modification is needed in CoffeeTags side. https://github.com/lukaszkorecki/CoffeeTags/pull/47.

And...now CoffeeTags can be run as a backend(I call xcmd, eXternal command.) Code is not merged yet. Much more cleanup are needed.

Look at --coffee-xcmd= in the ctags command line. It is something like ---regex. ---xcmd can be used to specify an external command for .

$ cat ~/var/ctags-github/xcmd-test.sh
./ctags -o /tmp/tags \
    --langdef=coffee \
    --langmap=coffee:.coffee \
    --coffee-xcmd=./run-coffeetags.sh 
    --fields='*' --file-scope=yes --extra=+fq '--*-kinds=*' \
    foo.coffee ant.c && cat /tmp/tags

$ cat run-coffeetags.sh
#!/bin/sh
RUBYLIB=/home/yamato/var/CoffeeTags/lib /home/yamato/var/CoffeeTags/bin/coffeetags --include-vars "$@"

$ cat foo.coffee 
# Taken from http://coffeescript.org/

class Animal
  constructor: (@name) ->

  move: (meters) ->
alert @name + " moved #{meters}m."

...

$ sh ~/var/ctags-github/xcmd-test.sh
!_TAG_FILE_FORMAT   2   /extended format; --format=1 will not append ;" to lines/
!_TAG_FILE_SORTED   1   /0=unsorted, 1=sorted, 2=foldcase/
!_TAG_PROGRAM_AUTHOR    Darren Hiebert  /dhiebert@users.sourceforge.net/
!_TAG_PROGRAM_AUTHOR!coffee Łukasz Korecki /lukasz@coffeesounds.com/
!_TAG_PROGRAM_NAME  Exuberant Ctags //
!_TAG_PROGRAM_NAME!coffee   CoffeeTags  //
!_TAG_PROGRAM_URL   http://ctags.sourceforge.net    /official site/
!_TAG_PROGRAM_URL!coffee    https://github.com/lukaszkorecki/CoffeeTags /GitHub repository/
!_TAG_PROGRAM_VERSION   Development //
!_TAG_PROGRAM_VERSION!coffee    0.3.1   //
AntParser   ant.c   /^extern parserDefinition* AntParser ()$/;" kind:function   line:32 language:C
ant.c   ant.c   1;" kind:file   line:1  language:C
constructor foo.coffee  /^  constructor: (@name) ->$/;" kind:function   line:4  language:coffee
def ant.c   /^  parserDefinition* const def = parserNew ("Ant");$/;"    kind:local  line:35 language:C
extensions  ant.c   /^  static const char *const extensions [] = { "build.xml", NULL };$/;" kind:local  line:34 language:C  file:
foo.coffee  foo.coffee  1;" kind:file   line:1  language:coffee
installAntRegex ant.c   /^static void installAntRegex (const langType language)$/;" kind:function   line:24 language:C  file:   signature:(const langType language)
move    foo.coffee  /^  move: (meters) ->$/;"   kind:function   line:6  language:coffee
move    foo.coffee  /^  move: ->$/;"    kind:function   line:10 language:coffee
move    foo.coffee  /^  move: ->$/;"    kind:function   line:15 language:coffee

[masatake]

popen version is works fine. However, like system(3), popen(3) is not secure enough. I will make my code private for awhile till my own pipe wrapper can be implemented.

[masatake]

Finally I wrote popen alternative named pcoproc. fork, execv and pipe are used in pcoproc. New ---xcmd is based on the pcoproc. A driver for coffeetags is added to our source tree. Unit test harness is updated for xcmd. Hacking guide is updated.

Now you can use ---xcmd as an alternative for ---regex.

[masatake]

Please, try this new feature.

[uri]

@masatake is this still relevant? I'm trying to get this working with coffee tags. I have

--langdef=coffee
--langmap=coffee:.coffee
--coffee-xcmd='coffeetags -R'
--fields='*' --file-scope=yes --extra=+fq '--*-kinds=*

In coffee.ctags but I'm getting the error

ctags: Unknown option: --coffee-xcmd

[uri]

Ah, I see it was remove in #1442 😭