search

rodjek / vim-puppet

Puppet niceties for your Vim setup
Apache License 2.0
502 stars 137 forks source link

Detect original filetype of an embedded puppet file #146

Closed shadowwa closed 3 months ago

shadowwa commented 1 year ago

vim-puppet is currently loading syntax/sh.vim and adding specific puppet region for highlighting.

I tried to detect more finely the original type by asking vim to do a filetypedetect on a filename without 'epp' extension. I also changed the &filetype value to originalfiletype.epuppet so various plugins like ALE or coc.nvim recognise the original filetype and allow linting or autocompletion I tried yaml, toml, various conf file and didn't find any drawback.

lelutin commented 8 months ago

thanks for working on this @shadowwa ! it was something that I wanted to get to for a while

I've tried the changes in this PR and so far the results are:

  1. the "sh" default syntax somehow does not seem to get loaded
  2. I tried opening a file name templates/config.inc.php.epp and the underlying type was not detected as php

should number 2 have figured out the type via the file name?

shadowwa commented 8 months ago

Strange, I tried with a minimal .vimrc to be sure that I haven't any other plugin or configuration interfering

call plug#begin()
Plug 'shadowwa/vim-puppet', { 'branch': 'detect_original_filetype','for': ['puppet', 'epuppet'] }
call plug#end()

and it's working fine

i initially used vim 9.0 but I have the same behaviour with vim 9.1 and neovim 0.7.2 and neovim 0.9.5

lelutin commented 8 months ago

hmm strange.. I've commented out everything in my .vimrc and kept just the plugin activation like in your example, and disabled all other loaded files except for plug.vim and I'm still seeing the same thing.

when I do :echo &ft I get epuppet.epuppet

I'm using vim 9.1 in debian sid.

I'll try and do some tracing of the filetype detection to see if I can understand what's happening

lelutin commented 8 months ago

err ok something's off on my side. I cloned your repo and switched to the branch for this PR and now it's working as expected. I'll see if I have some weird state in my clone of rodjek's repository

lelutin commented 8 months ago

ah! I found something out. vim 9.1 has something that was merged upstream and it's already defining the 'epuppet' filetype detection:

/usr/share/vim/vim91/filetype.vim:au BufNewFile,BufRead *.epp       setf epuppet

so the issue is not with your code but with this file short-circuiting things

lelutin commented 8 months ago

@shadowwa I took a peek at how subtype detection is currently done in upstream vim for eruby and I was able to replicate what it does for epuppet, squashing the bug that I have in vim 9.1

Basically the difference is that subtype detection is done inside ftplugin/$type.vim instead of inside ftdetect/. Also the type detection for eruby avoids calling out to doautocmd, I'm guessing that's in order to avoid type detection loops or weird conditions.

I've prepared a branch in my own fork that builds on top of your branch to get the final result that works for me in 9.1: https://github.com/lelutin/vim-puppet/tree/detect_epuppet_subtype

can I ask you to test if that works well for you as well in 9.0 ? if it does, we'll only need to add some unit tests for the new subtype detection and then we can merge this feature to master in this repo

shadowwa commented 8 months ago

The first version of the patch I wrote was also base on the eruby.vim but I found out that it was only taking into account the file extension and not the path or shebang as vim would do for files without .epp. for example: if opening a file templates/etc/apache/test.conf.epp, templates/etc/sudoers.d/test.epp or a perlscript with shebang but no extension, the eruby way would detect conf.epuppet or just epuppet instead of apache.epuppet, sudoers.epuppet and perl.epuppet respectively.

I tried to reproduce your error with my patch by using a fresh installed version of bookworm upgraded to sid but didn't succeeded. here are the diffents result I got:

debian bookworm vim 9.0.1378-2 ubuntu mantic vim 9.0.1672 debian sid vim 9.1.0016 debian bookworm neovim 0.7.2-7 ubuntu mantic neovim 0.9.5-6 mageia 10 vim 9.1.111 mageia 10 neovim 0.9.5-1
patch shad
perl.epp perl.epuppet perl.epuppet perl.epuppet perl.epuppet perl.epuppet perl.epuppet perl.epuppet
shell.epp epuppet epuppet.epuppet [^1] epuppet epuppet epuppet epuppet epuppet
shebanshell.epp sh.epuppet sh.epuppet sh.epuppet sh.epuppet sh.epuppet sh.epuppet sh.epuppet
shell.sh.epp sh.epuppet sh.epuppet sh.epuppet sh.epuppet sh.epuppet sh.epuppet sh.epuppet
scriptweb.php.epp php.epuppet php.epuppet php.epuppet php.epuppet php.epuppet php.epuppet php.epuppet
etc/apache/test.conf.epp apache.epuppet apache.epuppet apache.epuppet apache.epuppet apache.epuppet apache.epuppet apache.epuppet
patch lutin
perl.epp .epuppet [^2] .epuppet [^2] .epuppet [^2] .epuppet [^2] .epuppet [^2] epuppet [^2][^3] .epuppet [^2]
shell.epp .epuppet [^2] .epuppet [^2] .epuppet [^2] .epuppet [^2] .epuppet [^2] epuppet [^2][^3] .epuppet [^2]
shebanshell.epp .epuppet [^2] .epuppet [^2] .epuppet [^2] .epuppet [^2] .epuppet [^2] epuppet [^2][^3] .epuppet [^2]
shell.sh.epp sh.epuppet sh.epuppet sh.epuppet sh.epuppet sh.epuppet epuppet [^3] sh.epuppet
scriptweb.php.epp php.epuppet php.epuppet php.epuppet php.epuppet php.epuppet epuppet [^3] php.epuppet
etc/apache/test.conf.epp conf.epuppet conf.epuppet conf.epuppet conf.epuppet conf.epuppet epuppet [^2] conf.epuppet

[^2]: seem there is a uninitialized variable, I got the following error each time: "templates/perl.epp" 6L, 70B Error detected while processing BufRead Autocommands for ".epp"..FileType Autocommands for "epuppet"..function 4_lod_ft[4]..4_doautocmd[2]..FileType Autocommands for ""..function 7_LoadFTPlugin[18]..s cript /home/shad/.vim/plugged/vim-puppet/ftplugin/epuppet.vim: line 40: E121: Undefined variable: g:epuppet_default_subtype Press ENTER or type command to continue

[^3]: the vim packaged by mageia give a 'epuppet' filetype every time, the same version from appimage give same results than other vim versions

[^1]: vim is slow to start (3seconds) and if I tried the same vim version from appimage, vim start instantaneously and the filetype is epuppet like othe (neo)vim versions

lelutin commented 8 months ago

huh interesting, thanks for doing all of those tests on many different releases. it's kind of a shame how much effort we must put on the github CI in order to test in other releases

I

lelutin commented 8 months ago

Gah.. I hit something by mistake while typing. Didn't mean to close this.

I was going to say: I did some additional testing just now and somehow now I'm not getting epuppet.puppet with your code anymore. However, I can see that the tests re-ran because of my accidentally closing the PR and they are exhibiting what I used to see. e.g. the filetype detection ends up with epuppet.epuppet:

  Starting Vader: /home/runner/work/vim-puppet/vim-puppet/test/filetype/epuppet.vader
    (1/2) [EXECUTE] Filetype detection on a new empty file
    (1/2) [EXECUTE] (X) 'epuppet' should be equal to 'epuppet.epuppet'
    (2/2) [  GIVEN] epuppet file with leading tag
    (2/2) [EXECUTE] trigger filetype detection
    (2/2) [   THEN] should be detected as epuppet
    (2/2) [   THEN] (X) 'epuppet' should be equal to 'epuppet.epuppet'

So your code does a better job at detecting than mine, but the detection is flaky. I'm currently out of ideas for where to direct things. Do you have an idea for how we could make detection in your code more reliable, given the presence of filetype detection in upstream vim? maybe we could try to add sub-type detection in upstream vim directly?

shadowwa commented 8 months ago

while trying to complete vader tests, I saw that (even on the rodjek/vim-puppet master branch) all tests for file detection using

Given (buffer content):
  a buffer content

Execute (trigger filetype detection):
  file filename.extension
  edit

Then (should be detected as extension):
  AssertEqual &filetype, 'extension'

failed when using vim from latest debian, ubuntu or mageia (and neovim 9.5). Did you also noticed a failure of these tests on the rodjek/vim-puppet master branch?

Using vim appimage, I was able to see that while version 9.0.0260 was working fine, 9.0.0270 broke the tests. I suspected https://github.com/vim/vim/commit/2eddbacd6dc17c84e4bdc41e60e81949a36bb973 and tried to use true file and replacing the previous tests by

Execute (Load true file):
  edit test/test-files/filename.extension
  AssertEqual &filetype, 'extension'

I've created another branch to test and it seems to work for vim above 9.0.0270 and with neovim 9.5 https://github.com/shadowwa/vim-puppet/tree/test-with-files It wont solve detection problems from my code but at least, results from vader test and manual test should be more consistent. If you're ok with theses test, I will update them here while adding other filetype detection tests.

shadowwa commented 8 months ago

I've found another reason explaining the "flakiness" of our results: I assumed that loading the plugin by modifying rtp (as done in test/init.vim) or using plug-vim would be the same, but I was wrong:

To solve the problem, I just have to do the same thing that is done for .pp file: use au! instead of au.

Now the subfiletype is correctly detected on vim and nvim (at least up to v0.7.2 from mantic) and even solve the slow starting and epuppet.epuppet I got on vim native package of mantic. There is still a problem left: neovim from version 8.0 switched filedetection from filetype.vim to filetype.lua and the symptom of epuppet.epuppet detection persists (everything works fine when using vim-plug). I've yet have to find how to override the native detection for epp files.

shadowwa commented 7 months ago

I've finally solved the filetype detection for neovim >= 0.8.0. Had to redo the same detection in ftdetect/puppet.lua and disable the autocmd in ftdetect/puppet.vim to be able to override the native detection on recent neovim. The detection is now correct for vim and neovim (0.7.0, 0.8.0, 0.9.0 and 0.9.5) with or without plug-vim

lelutin commented 3 months ago

Hello @shadowwa

wow! thanks for pushing this forward. I really appreciate the work that you put into this (also it's definitely a feature that I want in this module). I'm sorry it took me so long before following up.. I kinda fell into a wormhole for a little while :|

(this is odd.. why didn't github trigger the CI on your latest pushes?..)

trying a new test run with your changes:

vim-puppet (git:pr/146) $ TESTVIM=vim ./test/run-tests.sh 
+ TEST_FILE=
+++ readlink -f ./test/run-tests.sh
++ dirname /home/gabster/.config/nvim/bundle/vim-puppet/test/run-tests.sh
+ SCRIPT_FOLDER=/home/gabster/.config/nvim/bundle/vim-puppet/test
+ '[' vim == vim ']'
+ RUNVIM=vim
+ cd /home/gabster/.config/nvim/bundle/vim-puppet/test/..
+ '[' '!' -d vader.vim ']'
+ '[' -z '' ']'
+ TEST_SUITE='test/**/*.vader'
+ vim -u test/init.vim -c 'Vader! test/**/*.vader'
Vim: Warning: Output is not to a terminal
Vader note: cannot print to stderr reliably/directly.  Please consider using Vim's -es/-Es option (mode=n).
VIM - Vi IMproved 9.1 (2024 Jan 02, compiled Jun 24 2024 14:19:35)
Included patches: 1-496
Modified by team+vim@tracker.debian.org
Compiled by team+vim@tracker.debian.org
Huge version without GUI.  Features included (+) or not (-):
+acl               +comments          +find_in_path      +lispindent        +multi_byte        +rightleft         +terminal          +wildignore
+arabic            +conceal           +float             +listcmds          +multi_lang        +ruby              +terminfo          +wildmenu
+autocmd           +cryptv            +folding           +localmap          -mzscheme          +scrollbind        +termresponse      +windows
+autochdir         +cscope            -footer            +lua               +netbeans_intg     +signs             +textobjects       +writebackup
-autoservername    +cursorbind        +fork()            +menu              +num64             +smartindent       +textprop          -X11
-balloon_eval      +cursorshape       +gettext           +mksession         +packages          +sodium            +timers            +xattr
+balloon_eval_term +dialog_con        -hangul_input      +modify_fname      +path_extra        -sound             +title             -xfontset
-browse            +diff              +iconv             +mouse             +perl              +spell             -toolbar           -xim
++builtin_terms    +digraphs          +insert_expand     -mouseshape        +persistent_undo   +startuptime       +user_commands     -xpm
+byte_offset       -dnd               +ipv6              +mouse_dec         +popupwin          +statusline        +vartabs           -xsmp
+channel           -ebcdic            +job               +mouse_gpm         +postscript        -sun_workshop      +vertsplit         -xterm_clipboard
+cindent           +emacs_tags        +jumplist          -mouse_jsbterm     +printer           +syntax            +vim9script        -xterm_save
-clientserver      +eval              +keymap            +mouse_netterm     +profile           +tag_binary        +viminfo           
-clipboard         +ex_extra          +lambda            +mouse_sgr         -python            -tag_old_static    +virtualedit       
+cmdline_compl     +extra_search      +langmap           -mouse_sysmouse    +python3           -tag_any_white     +visual            
+cmdline_hist      -farsi             +libcall           +mouse_urxvt       +quickfix          +tcl               +visualextra       
+cmdline_info      +file_in_path      +linebreak         +mouse_xterm       +reltime           +termguicolors     +vreplace          
   system vimrc file: "/etc/vim/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
 3rd user vimrc file: "~/.config/vim/vimrc"
      user exrc file: "$HOME/.exrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -Wdate-time -g -O2 -Werror=implicit-function-declaration -ffile-prefix-map=/build/reproducible-path/vim-9.1.0496=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -DSYS_VIMRC_FILE=\"/etc/vim/vimrc\" -DSYS_GVIMRC_FILE=\"/etc/vim/gvimrc\" -D_REENTRANT -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 
Linking: gcc -Wl,-E -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim -lm -ltinfo -lselinux -lsodium -lacl -lattr -lgpm -L/usr/lib -llua5.1 -Wl,-E -fstack-protector-strong -L/usr/local/lib -L/usr/lib/x86_64-linux-gnu/perl/5.38/CORE -lperl -ldl -lm -lpthread -lcrypt -L/usr/lib/python3.12/config-3.12-x86_64-linux-gnu -lpython3.12 -ldl -lm -L/usr/lib/x86_64-linux-gnu -ltcl8.6 -ldl -lz -lpthread -lm -lruby-3.1 -lm -L/usr/lib 

Starting Vader: 17 suite(s), 47 case(s)
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/filetype/epuppet.vader
    (1/7) [EXECUTE] Filetype detection on a new empty file
    (1/7) [EXECUTE] (X) 'epuppet' should be equal to 'epuppet.epuppet'
    (2/7) [EXECUTE] epuppet test_with_leading_tag
    (2/7) [EXECUTE] (X) 'epuppet' should be equal to 'epuppet.epuppet'
    (3/7) [EXECUTE] epuppet perl with shebang
    (3/7) [EXECUTE] (X) 'perl.epuppet' should be equal to 'epuppet.epuppet'
    (4/7) [EXECUTE] epuppet shell with shebang
    (4/7) [EXECUTE] (X) 'sh.epuppet' should be equal to 'epuppet.epuppet'
    (5/7) [EXECUTE] epuppet shell with extension
    (5/7) [EXECUTE] (X) 'sh.epuppet' should be equal to 'epuppet.epuppet'
    (6/7) [EXECUTE] epuppet php with extension
    (6/7) [EXECUTE] (X) 'php.epuppet' should be equal to 'epuppet.epuppet'
    (7/7) [EXECUTE] epuppet apache conf with path and extension
    (7/7) [EXECUTE] (X) 'apache.epuppet' should be equal to 'epuppet.epuppet'
  Success/Total: 0/7
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/filetype/puppet.vader
    (1/1) [EXECUTE] Load simple puppet file
    (1/1) [   THEN] Detected simple.pp as a puppet file
  Success/Total: 1/1
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/filetype/puppetfile.vader
    (1/1) [EXECUTE] Load Puppetfile
    (1/1) [   THEN] Detected Puppetfile as a ruby file
  Success/Total: 1/1
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/format/hashrocket.vader
    (1/2) [  GIVEN] simple resource with autoformat while writing
    (1/2) [     DO] format resource
    (1/2) [ EXPECT] formated resource
    (2/2) [  GIVEN] simple resource with gq
    (2/2) [     DO] format resource
    (2/2) [ EXPECT] formated resource
  Success/Total: 2/2
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/format/textwidth.vader
    (1/2) [  GIVEN] long line
    (1/2) [     DO] format all text
    (1/2) [ EXPECT] nothing changed
    (2/2) [  GIVEN] long line
    (2/2) [     DO] format all text with textwidth set
    (2/2) [ EXPECT] comment is wrapped into more lines
  Success/Total: 2/2
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/indent/basic.vader
    (1/3) [  GIVEN] class with includes
    (1/3) [     DO] full text indent with '='
    (1/3) [ EXPECT] indented class
    (2/3) [  GIVEN] class with includes
    (2/3) [     DO] full text indent with 'gq' in Visual Mode
    (2/3) [ EXPECT] indented class
    (3/3) [  GIVEN] class with includes and resources
    (3/3) [     DO] full text indent by gq with motion
    (3/3) [ EXPECT] indented class
  Success/Total: 3/3
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/indent/comments_strings.vader
    (1/4) [  GIVEN] comments before closing brace
    (1/4) [     DO] full text indent with '='
    (1/4) [ EXPECT] closing brace back aligned with start of resource block
    (2/4) [  GIVEN] closing brace after comment that contains closing brace
    (2/4) [     DO] full text indent with '='
    (2/4) [ EXPECT] closing brace back aligned with start of resource block
    (3/4) [  GIVEN] multi-line string
    (3/4) [     DO] full text indent with '='
    (3/4) [ EXPECT] multi-line string does not move and line after resumes indentation
    (4/4) [  GIVEN] square bracket and hash sign
    (4/4) [     DO] New line
    (4/4) [ EXPECT] indent stays the same
  Success/Total: 4/4
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/indent/include.vader
    (1/3) [  GIVEN] include of multiple classes over multiple lines
    (1/3) [     DO] full text indent with '='
    (1/3) [ EXPECT] indented include
    (2/3) [  GIVEN] resource with param named include
    (2/3) [     DO] full text indent with '='
    (2/3) [ EXPECT] properly aligned parameters
    (3/3) [  GIVEN] resource with param named include contained in class
    (3/3) [     DO] full text indent with '='
    (3/3) [ EXPECT] second contained resource should align with the first one
  Success/Total: 3/3
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/syntax/array.vader
    (1/4) [  GIVEN] simple array
    (1/4) [EXECUTE] syntax is good
    (2/4) [  GIVEN] empty array
    (2/4) [EXECUTE] syntax is good
    (3/4) [  GIVEN] nested array
    (3/4) [EXECUTE] syntax is good
    (4/4) [  GIVEN] array with nested hash
    (4/4) [EXECUTE] syntax is good
  Success/Total: 4/4
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/syntax/boolean.vader
    (1/2) [  GIVEN] true
    (1/2) [EXECUTE] syntax is good
    (2/2) [  GIVEN] false
    (2/2) [EXECUTE] syntax is good
  Success/Total: 2/2
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/syntax/epuppet.vader
    (1/3) [  GIVEN] template with litteral content puppet tags
    (1/3) [EXECUTE] litteral content syntax must be correct
    (2/3) [  GIVEN] template with litteral content puppet tags
    (2/3) [EXECUTE] epp delimiter syntax must be correct
    (3/3) [  GIVEN] template with litteral content puppet tags
    (3/3) [EXECUTE] puppet syntax must be correct
  Success/Total: 3/3
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/syntax/fold.vader
    (1/1) [  GIVEN] basic class
    (1/1) [EXECUTE] fold level # in code block
  Success/Total: 1/1
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/syntax/number.vader
    (1/4) [  GIVEN] hexidecimal
    (1/4) [EXECUTE] syntax is good
    (2/4) [  GIVEN] integer
    (2/4) [EXECUTE] syntax is good
    (3/4) [  GIVEN] octal
    (3/4) [EXECUTE] syntax is good
    (4/4) [  GIVEN] floating point
    (4/4) [EXECUTE] syntax is good
  Success/Total: 4/4
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/syntax/regex.vader
    (1/1) [  GIVEN] division operation
    (1/1) [EXECUTE] syntax is good
  Success/Total: 1/1
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/syntax/strings.vader
    (1/2) [  GIVEN] heredoc with trailing comma on first line
    (1/2) [EXECUTE] syntax is undisturbed by trailing comma or apostrophe
    (2/2) [  GIVEN] heredoc with simple closing symbol
    (2/2) [EXECUTE] closing symbol properly ends syntax group
  Success/Total: 2/2
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/syntax/variable.vader
    (1/3) [  GIVEN] local variables
    (1/3) [EXECUTE] syntax is good
    (2/3) [  GIVEN] global scope variables
    (2/3) [EXECUTE] syntax is good
    (3/3) [  GIVEN] out of scope qualified name variables
    (3/3) [EXECUTE] syntax is good
  Success/Total: 3/3
  Starting Vader: /home/gabster/.config/nvim/bundle/vim-puppet/test/syntax/variable_in_string.vader
    (1/4) [  GIVEN] interpolated top scope variable
    (1/4) [EXECUTE] syntax is good
    (2/4) [  GIVEN] local variable
    (2/4) [EXECUTE] syntax is good
    (3/4) [  GIVEN] unenclosed local variable
    (3/4) [EXECUTE] syntax is good
    (4/4) [  GIVEN] unenclosed top scope variable
    (4/4) [EXECUTE] syntax is good
  Success/Total: 4/4
Success/Total: 40/47 (assertions: 252/259)
Elapsed time: 1.90 sec.
+ vader_exit=1
+ '[' -n '' ']'
+ exit 1
lelutin commented 3 months ago

small parenthesis: I've just tried reactivating neovim tests in CI and they don't cause coredumps anymore! :tada: so we can have neovim tests in CI again. interestingly, some tests are currently failing on neovim on master and this branch is fixing them!

see: https://github.com/rodjek/vim-puppet/actions/runs/10589033615/job/29342493381?pr=147

I can reproduce the neovim fails locally with nvim 0.9.5

so your branch is making some part of the code better. we just need to figure out what's happening with vim 9.1 and sub-type detection

lelutin commented 3 months ago

ok this is not super useful as a "how do we fix this" but I can confirm that the test failures on vim are caused by this line:

https://github.com/vim/vim/blob/master/runtime/filetype.vim#L1847

If I comment it out, the tests all pass with vim 9.1 (current package on debian sid).

to see a bit more what was happening, I've added echo &ft in ftdetect/puppet.vim above and below the line execute 'doautocmd filetypedetect [...] and then opened up a .epp file with vim. both before and after that line &filetype was set to epuppet. from the subsequent lines it looks like the function expects filetype to have been changed into the subtype, so something's not going according to plan there.

lelutin commented 3 months ago

@shadowwa I did a bit more digging around and here's what I can find:

So the code proposed in this PR is novel and we should find a way to make it work!

If we want to think about the future though, I think that we should work with both upstreams, vim and neovim, to get the subtype detection for epuppet merged there once we have something that works. We could probably propose the code for neovim now.

note: I'm not willing to wait for the above to happen before we merge your code in. I think we should get your work finalized and merged into the vim-puppet plugin, and then work on pushing the changes to the upstreams.

If we're able to get the change merged into the upstreams, then we could add a version fence around filetype detection in this plugin to let the builtin type detection happen starting with the versions where the work got merged in. Then in a couple years once we decide to stop supporting versions of vim and neovim older than those points, we'll be able to trop the filetype detection code from this module.

Do you think this idea is worth moving towards? Would you like to open issues and/or pull requests on neovim and eventually vim ? or would you prefer to let me handle this? I don't mind opening issues, but I want to leave the you the option of being the point of contact in vim/neovim wrt the changes to you if you so desire.

lelutin commented 3 months ago

@shadowwa it seems as though the line execute 'doautocmd filetypedetect BufRead [...] is not doing anything because it ends up calling setf sometype and that won't work since the filetype was already set to epuppet at that point.

if I add set verbose=9 just above the doautocmd line I get this when opening the file test/test-files/test_php_with_extension.php.epp:

Executing BufRead Autocommands for "*.php"
autocommand setf php

see: https://vimhelp.org/options.txt.html#%3Asetf

unfortunately, I have not found a way to "unset" filetype in order to let the setf do its thing.

I've reworked the code for this merge request and yet again moved closer to how things are done for eruby. It only detects the subtype by the filename's extension but not via the shebang or the path to the file. This PR has been open for a long time now and the state of my rework is better than nothing.

I'll open up an issue and tag you there so that we can follow up on the missing bits: reusing builtin filetype detection to get content parsing and discovery through the path to files.

shadowwa commented 2 months ago

Testing the latest version shows that if I got the highlight correctly detected based on the subtype file extension before the .epp, I'm losing other niceties like linting of the subtype by ALE or activation of the subtype LSP by coc.nvim. re adding the

let &filetype=b:epuppet_subtype .'.epuppet'

in syntax/epuppet.vim does the trick but as you mentioned in 4938353, setting the filetype in syntax file should not be the way.

Weirdly, I was using my branch almost every days with several vim version, and the detection was always OK. And for nvim, I didn't saw that it was updated to 0.10.1 and testing now show that my branch didn't work anymore. I don't really get why we have opposite results regarding vim vs. nvim.

I'll try to see if I can find a way to get the coc/ale works with highlight while setting the filetype on the ftdetect place, and why we get opposite result with vim/nvim.

A working subtype detection should really be nice on upstream vim/nvim.