`sequence//3` doesn't like trailing separators

[pkoch]

% swipl -g examples sequence_derp.pl
:- use_module(library(dcg/basics)).
:- use_module(library(dcg/high_order)).

one_line([Op, N]) --> string(Op), white, number(N).

sequence_phrases(L) --> sequence(one_line,"\n", L), ("\n"|[]).

manual_phrases([]) --> [].
manual_phrases([H|T]) --> one_line(H), !,("\n"|[]), manual_phrases(T).

other_line([Op, N]) --> number(N), white, string(Op).
two_sequences(L) -->
  sequence(one_line,"\n", LA),
  (([], {LB = []})|("\n", sequence(other_line,"\n", LB))),
  {append(LA, LB, L)}.

exercise(Dcg,Codes):-
  Goal =.. [Dcg, _L],
  phrase(Goal, Codes).

examples:-
  NoNL = `a 1\nb 2`,
  WithNL = `a 1\nb 2\n`,

  exercise(sequence_phrases, NoNL),
  \+ exercise(sequence_phrases, WithNL),
  exercise(manual_phrases, NoNL),
  exercise(manual_phrases, WithNL),

  exercise(two_sequences, `a 1\nb 2`),
  \+ exercise(two_sequences, `a 1\nb 2\n1 a\n2 b`),
halt.

I'd very much expect that sequence should work even if there's more lines in the file.

It might be easy to dismiss this over "\n at the end of file is an edge case", but this means I can't have any more lines in the file (like in two_sequences).

[JanWielemaker]

sequence//3 matches Element?, (Sep,Element)* according to the docs. The parser commits on each Sep. We could of course not do so, but that would create huge amounts of choice points making the parser nice for small demos but useless for real input. If you know a better solution that does avoid piling up choice points, please share it. Else, please close.

There is a related discussion on https://swi-prolog.discourse.group/t/confusing-behavior-of-sequence-3-and-sequence-5-from-dcg-high-order/1607

[pkoch]

Thanks for the pointer to the current discussion!

I don't think that Element?, (Sep,Element)* fully describes what's going on. I think that adding this caveat in the docs would probably be ideal for now.

Since there's a bunch more juicy info on that thread, and this is more of a known problem that's requiring deliberation, I'd argue it's not really an issue. Having this in the issues such that other folks can follow the trail already seems good enough. Closing!