textmate support config TokenizerOption lineLimit

502647092 commented 5 years ago

Description

Reproduction Steps

create a html file

        <div class="img-border">
          <a href="#">
            <div class="img">
              <img src="https://gitee.com/uploads/61/632261_smallweigit.jpg">
            </div>
          </a>
          <img
            alt="用户等级"
            src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAsCAMAAAApWqozAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAMAUExURUxpcf9dfP9aef9be/9dfP9gfv9hf/9kgP9eff+bm/9dff8yWf84XP82XP82XP84Xv9jgv84Xv83XP8xVv9BYv9Ia/89Yf89Y/8uVf9Fav8/Y/9igP9Hav8vV/9lgv8uVf8xW/8uVf9kgv8uVf8vWP9jgv8uWP9Vdf+A//9mhf9jgf9lgv9lgf9kgv9Wdf9lgv9Rcf9Nb/9Ocf9be/8xWP8yWP9AYP9gff9ig/9mgP9ff/85YP81W/8yWf86X/85Y/9aef9ZeP8xWP9AY/9Faf8xWf8zWv89Yf81W/8/ZP9JbP85cf9YeP9EaP9Zd/9FZ/87Yf86X/9Lbf9Xd/8uVv9Kbf8sVv9AZP9DZv9JbP9Ca/9Gaf9Daf9Wd/9Tc/8uVf9Nbf8tVP9Qdf9TdP9TdP9Mbf////9FaP9EZ/9Ja/9Lbf9aef9Wdv9ZeP9Gaf82XP9PcP9bev80Wv9KbP9BZf9AZP9efP86X/89Yv9fff8wV/9ce/9Xd/9Hav9gfv9UdP9Qcf9Vdf8xWP9Sc/83XP81W/83Xf88Yf8yWf87YP8+Yv9Nbv9Rcv9Yd/9Iav9CZv9Mbv8vVv8zWf8/Y/9Tc/9OcP9if/9DZv/+/v9Ob/9hf/85Xv84Xv9jgf8tVf9de/9dfP85X/8+Y/9TdP9Nb/9YeP9Ia/9JbP9Mbf9Haf9Wdf8zWv88YP9Scv+wv/9igP9cev80W/9CZf9BZP91j//3+f/Q2f9DZ//T3P97lf8yWP/r7//6+//W3f9Rcf+ltv9hfv94kf9+lv/g5v/z9f9eff/Q2P95kv+ouf/t8P++yv/m6//d5P/G0f/a4f9ffv+ywf+8yP+2w/+is/9Kbf+jtP+Vqf+fsf+cr/+OpP/S2v+Nov/V3f/5+v9lg//Dzv/j6P/J0//w8//M1v/G0P+Inv+Mof/s8P+/y//X3v+puf/h5v/v8v+5xv+ZrP+vvv/19/+Em//09v/9/f9ohf+4xf+0wv9phf/L1P9yjf+QpP+KoP+Spv+svP+ru/+Yq/TCRPAAAABmdFJOUwD+/lSgTrYEfAEsoxvxOCg/v9xRE+uT+3shf8jR0vPDD3RISCWeLtICOurApoRR2fCpi37GPQjyJyjzeOfL8RL19sckuD/MFejR9wm9cbqe8nveSf3cYs5u9B+2nEem/MRhRqNEwud7RXQAAARfSURBVDjLfZV3WFNXGMZv7aCtdlhnrbbWUUe1Vbv30Nq9996hCaE3hOxAck0gIWElBlISCAYoQhUMWCgCClopIFoVaRG02Kq4tdDSPf2+c26SCxLPn/f+nvd5z3fe8x6GGbyi3lu08J3nHhs3ecnTT17JnG1ddMNbCzhPqkMiTvw4+qNPFI9OWhyRfX6W0czJslMl8eKMz6JrMhVK14TxDw+LXvzEqmVZ5jRZ9qcAo3Lm50q7xvrMs1Fnsuc+kC5dVlTegDbixSuJDWWhtcyWO3rsUPamEcVSKVGWOUol4gyA1yqUdrtVrbZccdngnZ0XFxOLymaOIzbAM1HWlNkK8thLLhXCF+oBTpAai7LSZB7YYAnxrHAVgnKuhU1+ZFSYPf+cFRVE2ViOGwTPGYlE2WW1qgvyLMnypY8H2Tvu/Ioo89NIlXxTEp6GusDCspUpI4PwBVrVCmIDYE4mSy2l08hcqyy022CDoLza9BBlLzdo/aCcjp7NaZyHn3MNetZYbbl5LCtPMT1I4asMBtW+vXsP//QLeoY5SySoXEM9K9K1Wq1kdZ1uDLLT3F6t3zcggnXCbW4Az474lSHPu3vwx+/LTbopCM/0GsBzM34TbUnAQxHMuZV83i5fnqJzzmCY2XMDYENfRaRFbRx6pqmDE9zXvx6/t8hB2Xktw8zxeTeCsp5K96i4bEdpfAnOGVLXdqCpev3ANrYSla9hmPkBt3eHSh/HS7cL8+zbWr/lZFN/LU5D53xhKjPP5wbP+ooYKl0dK8jzms0/bK0/sgHmjMr51zHX+wIIx8VsyiH0b+E8qzZs3wZ4S1KyPKUO4OnMjWBDq/LDCX5B4P5VoTx3NK8D/A81OUGAX2Kuhg0a/HDcxbx0XzDPsV/vPwj4JksSC7DJmT8RYN5GbPoeAjcV8XPuPPp97/6DbZo8FjyjjYlow6ClqWuh0q00z9Jdv7YDXoV5rsRp5L+LG4Tjpnmm0gc4kudDfTt3nWrfaSd5JtOYzswLeA07+DzXEljUgXnOamz9D/A/+TybyOjm+9wbQ3leRxNSCtOo3d3RePzfRlcwzzrn21OZOYFAOM+89HFxYkLn/12AV4XybMLjnj3XC0GCORfjHaTSA11Vf3f/0/lj16logMno6kiQmJmY5+Ad/JJKD/T+dehwd8exPk0ZvbCQOt0MDD/JcwVRLjJ/R2lRTv2JkznVEg2WDNiAaUwJXStUlkqNxnJemq41LquNegblMfTCkjzztzvtWJitlkEV2AosScmQ55uDVYB5RhtGo7lBIN3nckF95VosOI1bePj22zDPxaCM/dwbZOsdSpcGewOzMVJQX4JGkjbzrI80EhQj5PmeWwXF6OdbFPvZ073n557N33qC/QwnePcoYY1C5Ubu56TBlQvaIyL385Ayx2fi/kj9PPq+YR6gp4bt53vvihr2vXpz1hn9PGH82MiP5vsLhP388qSzP7JRHyxa+OGLr46b/Pobr7w29OdpmH8aJmElL+MAAAAASUVORK5CYII="
            class="img-v"
          >
        </div>

OS and Theia version: latest next version

Diagnostics:

kittaakos commented 5 years ago

According to the default TextMate tokenizer option, the line limit is 400 characters. If one of the lines has exceeded this limit, the TextMate tokenizer in the subsequent lines will be broken.

502647092 commented 5 years ago

how to config this prop

akosyakov commented 5 years ago

It does not look like it can be configured by a user only by a language contributor: https://github.com/theia-ide/theia/blob/5862b11c5809382e62ea9ea82e8ab03b14ad4a58/packages/monaco/src/browser/textmate/textmate-registry.ts#L21

We could introduce an option to override it per a language or file matching that a user can work it around.

eclipse-theia / theia

textmate support config TokenizerOption lineLimit #4765

Description

Reproduction Steps