PHP Syntax Color Error

[mahendra232]

Issue Type: Bug

When i want to use trying using JS on HEREDOC and set the mark to "JS", all syntax color below that HEREDOC become error...

Example:

$jsfunc=<<<JS
                $(document).ready(function(){
                    ...
                });
JS;

//Below this line the Syntax color become error

.....

VS Code version: Code 1.45.1 (5763d909d5f12fe19f215cbfdd29a91c0fa9208a, 2020-05-14T08:27:35.169Z) OS version: Windows_NT x64 6.3.9600

System Info

|Item|Value| |---|---| |CPUs|Intel(R) Celeron(R) CPU N2920 @ 1.86GHz (4 x 1867)| |GPU Status|2d_canvas: enabled
flash_3d: enabled
flash_stage3d: enabled
flash_stage3d_baseline: enabled
gpu_compositing: enabled
multiple_raster_threads: enabled_on
oop_rasterization: disabled_off
protected_video_decode: unavailable_off
rasterization: enabled
skia_renderer: disabled_off_ok
video_decode: enabled
viz_display_compositor: enabled_on
viz_hit_test_surface_layer: disabled_off_ok
webgl: enabled
webgl2: enabled| |Load (avg)|undefined| |Memory (System)|3.75GB (1.22GB free)| |Process Argv|| |Screen Reader|no| |VM|0%|

Extensions (2)

Extension|Author (truncated)|Version ---|---|--- php-debug|fel|1.13.0 vscode-counter|uct|1.3.5

[roblourens]

Works for me. Can you share the full text of the file?

[mahendra232]

Works for me. Can you share the full text of the file?

Here is:

<?php
defined('BASEPATH') OR exit('No direct script access allowed');

class Post extends CI_Controller {

    public function __construct(){
        parent::__construct();
        $this->load->helper('form');
        $this->load->library('form_validation');
        $this->load->model('comp');
        $this->login_check();
    }

    protected function login_check(){
        if(!isset($_SESSION['ans_logged_in'])){
            //No Access
            //Redirect to main page
            header('location: '.$this->comp->url_hash('?'));
            die();
        }
    }

    protected function captcha_generate(){
        $_SESSION['ans_captcha_pub']=$this->comp->captcha();
    }

    public function index(){
        header('location: '.$this->comp->url_hash('post/add_post?'));
        die(); 
    }

    public function write_post(){
        //Check if has previous session
        if(isset($_SESSION['ans_draft_post'])){
            //Set JS
            $jsfunc=<<<JS
                $(document).ready(function(){
                    prevSessConf({$_SESSION['ans_draft_post']});
                });
            JS;

            //Adding JS
            $footer['js_line']=array($jsfunc);
        }

        //Generate Captcha
        $this->captcha_generate();

        //load json config
        $header['site_config']=$this->comp->json_config('site_settings.json');

        //load component
        $header['page_title']='Posting';
        $footer['add_js']=array(
            'https://cdn.tiny.cloud/1/i7i2596njvqi9osf3i50q92tnxpasr36zo571973zz3g4lm7/tinymce/5/tinymce.min.js',
            base_url('res/js/editor.js')
        );

        $body['category_list']=$this->comp->category_select('all');

        //Load view
        $this->load->view('home/header',$header);
        $this->load->view('post/write_post',$body);
        $this->load->view('home/footer',$footer);
    }

    public function add_post(){
        //Check captcha
        if($_POST['captcha'] !== $_SESSION['ans_captcha_pub']['real']){
            echo 'Captcha salah, mohon ulangi kembali';
            die();
        }

        $config=array(
            array(
                'field' => 'title',
                'label' => 'Judul',
                'rules' => 'required'
            ),
            array(
                'field' => 'content',
                'label' => 'Kontent',
                'rules' => 'required'
            ),
            array(
                'field' => 'category',
                'label' => 'Kategori',
                'rules' => 'required'
            ),
            array(
                'field' => 'captcha',
                'label' => 'Captcha',
                'rules' => 'required'
            ),
            array(
                'field' => 'tinymce',
                'label' => 'TinyMCE',
                'rules' => 'required',
                'errors' => array(
                    'required' => 'Anda hanya dapat menggunakan TinyMCE untuk dapat membuat sebuah postingan.',
                ),
            ),
        );

        //Set up rules
        $this->form_validation->set_rules($config);

        //Run Validation
        if ($this->form_validation->run() == FALSE){
            //Throwback error
            echo validation_errors('<p style="font-weight: bold;">','</p>');
            die();
        } else{
            //Set default Variable for Featured Image
            $featured_image=NULL;

            if(is_uploaded_file($_FILES['featured_image']['tmp_name'])){

                //Set Unique Files Name
                $files_info=pathinfo($_FILES['featured_image']['name']);
                $files_ext='.'.$files_info['extension'];
                $files_name=md5($files_info['filename']).'_'.time().$files_ext;

                //Upload Config
                $file_config=array(
                    'file_name' => $files_name,
                    'remove_space' => TRUE,
                    'upload_path' => FCPATH.'assets/img/featured_image/',
                    'allowed_types' => 'jpg|png'
                );

                //Load library and config
                $this->load->library('upload', $file_config);

                //Run Validation
                if(!$this->upload->do_upload('featured_image')){
                    //Send error info
                    echo $this->upload->display_errors('<p style="font-weight: bold;">','</p>');
                    die();
                } else{
                    //Set New Variable
                    $featured_image=$files_name;
                }

            }

            //Select category id
            if(isset($_POST['sub_category'])){
                $cat_id=$_POST['sub_category'];
            } else{
                $cat_id=$_POST['category'];
            }

            //Make permalink from title
            $slug=preg_replace('/[^A-Za-z0-9-]+/', '-', $_POST['title']);
            $slug_permalink=$slug.'_'.mt_rand(100,1000);
            $permalink=strtolower($slug_permalink);

            //Insert Post to Database
            $post_data=array(
                'title' => strip_tags($_POST['title']),
                'content' => $_POST['content'],
                'permalink' => $permalink,
                'image_cover' => $featured_image,
                'posted_by' => $_SESSION['ans_uid'],
                'status' => 'approval',
                'category' => $cat_id
            );
            $this->db->insert('post',$post_data);

            //Add point:s
            /*
                Add pending point to user for post contribution rewards
            */
            //Add point:e

            //Unset SESSION from draft
            unset($_SESSION['ans_draft_post']);
            unset($_SESSION['ans_draft_img']);

            echo 'done';
            die();
        }
    }

    public function draft_post(){
        //Check captcha
        if($_POST['captcha'] !== $_SESSION['ans_captcha_pub']['real']){
            echo 'Captcha salah, mohon ulangi kembali';
            die();
        }

        $config=array(
            array(
                'field' => 'title',
                'label' => 'Judul',
                'rules' => 'required'
            ),
            array(
                'field' => 'content',
                'label' => 'Kontent',
                'rules' => 'required'
            ),
            array(
                'field' => 'category',
                'label' => 'Kategori',
                'rules' => 'required'
            ),
            array(
                'field' => 'captcha',
                'label' => 'Captcha',
                'rules' => 'required'
            ),
            array(
                'field' => 'tinymce',
                'label' => 'TinyMCE',
                'rules' => 'required',
                'errors' => array(
                    'required' => 'Anda hanya diperbolehkan menggunakan Edito TinyMCE untuk dapat membuat konten.',
                ),
            ),
        );

        //Set up rules
        $this->form_validation->set_rules($config);

        //Run Validation
        if ($this->form_validation->run() == FALSE){
            //Throwback error
            echo validation_errors('<p style="font-weight: bold;">','</p>');
            die();
        } else{
            //Set default Variable for Featured Image
            $featured_image=NULL;

            if(is_uploaded_file($_FILES['featured_image']['tmp_name'])){

                //Set Unique Files Name
                $files_info=pathinfo($_FILES['featured_image']['name']);

                //Check if file is same as before
                if(isset($_SESSION['ans_draft_img'])){
                    if($_FILES['featured_image']['name'] == $_SESSION['ans_draft_img']['name']){
                        $featured_image=$_SESSION['ans_draft_img']['name_encrypt'];
                        goto dHasSameImg;
                    }
                }

                $files_ext='.'.$files_info['extension'];
                $files_name=md5($files_info['filename']).'_'.time().$files_ext;

                $_SESSION['ans_draft_img']=array(
                    'name' => $_FILES['featured_image']['name'],
                    'name_encrypt' => $files_name
                );

                //Upload Config
                $file_config=array(
                    'file_name' => $files_name,
                    'remove_space' => TRUE,
                    'upload_path' => FCPATH.'assets/img/featured_image/',
                    'allowed_types' => 'jpg|png'
                );

                //Load library and config
                $this->load->library('upload', $file_config);

                //Run Validation
                if(!$this->upload->do_upload('featured_image')){
                    //Send error info
                    echo $this->upload->display_errors('<p style="font-weight: bold;">','</p>');
                    die();
                } else{
                    //Set New Variable
                    $featured_image=$files_name;
                }

            }

            dHasSameImg:

            //Select category id
            if(isset($_POST['sub_category'])){
                $cat_id=$_POST['sub_category'];
            } else{
                $cat_id=$_POST['category'];
            }

            //Make permalink from title
            $slug=preg_replace('/[^A-Za-z0-9-]+/', '-', $_POST['title']);
            $slug_permalink=$slug.'_'.mt_rand(100,1000);
            $permalink=strtolower($slug_permalink);            

            if(isset($_SESSION['ans_draft_post'])){
                //Code for draft session

                //Updating data
                $post_data=array(
                    'title' => strip_tags($_POST['title']),
                    'content' => $_POST['content'],
                    'permalink' => $permalink,
                    'image_cover' => $featured_image,
                    'category' => $cat_id
                );
                $this->db->where('id', $_SESSION['ans_draft_post']);
                $this->db->update('post', $post_data);

                //Send feedback to ajax
                echo 'done';
                die();
            }

            //Insert Post to Database
            $post_data=array(
                'title' => strip_tags($_POST['title']),
                'content' => $_POST['content'],
                'permalink' => $permalink,
                'image_cover' => $featured_image,
                'posted_by' => $_SESSION['ans_uid'],
                'status' => 'draft',
                'category' => $cat_id
            );
            $this->db->insert('post',$post_data);

            //Set Session for draft
            $_SESSION['ans_draft_post']=$this->db->insert_id();

            echo 'done';
            die();
        }
    }

    public function tinymce_upload() {
        $url = array( base_url() );

        reset($_FILES);
        $temp = current($_FILES);

        if (is_uploaded_file($temp['tmp_name'])) {
            if (preg_match("/([^\w\s\d\-_~,;:\[\]\(\).])|([\.]{2,})/", $temp['name'])) {
                header("HTTP/1.1 400 Invalid file name,Bad request");
                return;
            }

            // Validating File extensions
            if (! in_array(strtolower(pathinfo($temp['name'], PATHINFO_EXTENSION)), array(
                "gif",
                "jpg",
                "png",
                "bmp"
            ))) {
                header("HTTP/1.1 400 Not an Image");
                return;
            }

            $md5=md5($temp['name']);
            $ext=pathinfo($temp['name'], PATHINFO_EXTENSION);
            $enkripsi=$md5.'_'.time().'.'.$ext;
            $fileName = "assets/img/post_body/" . $enkripsi;
            move_uploaded_file($temp['tmp_name'], $fileName);

            // Return JSON response with the uploaded file path.
            echo json_encode(array(
                'file_path' => base_url($fileName)
            ));
            die();
        }
    }

    public function edit(){
        /* Edit Post Page */

        //Check if pid isset
        if(!isset($_GET['pid'])){
            header('location: '.base_url('user/link_invalid'));
            die();
        }

        $pid = $_GET['pid'];
        $pdata = $this->db->query("SELECT * FROM post WHERE id=$pid")->row();
        echo '<pre>';
        print_r($pdata);
        echo '</pre>';
        die();
    }

    public function edit_preview(){
        /* Edit Preview Page */

        //Check if pid isset
        if(!isset($_SESSION['ans_draft_post'])){
            header('location: '.base_url('user/link_invalid'));
            die();
        }

        $pid = $_SESSION['ans_draft_post'];
        $pdata = $this->db->query("SELECT * FROM post WHERE id=$pid")->row();
        echo '<pre>';
        print_r($pdata);
        echo '</pre>';
        die();
    }

    public function reset_post_session(){
        //Unset SESSION from draft
        unset($_SESSION['ans_draft_post']);
        unset($_SESSION['ans_draft_img']);

        echo 'done';
        die();
    }

}

[mahendra232]

Post.php.zip

[mahendra232]

I know i can fix that error by using another HEREDOC "mark"... but, i'm just curious what causing that error :smile:

[roblourens]

I can repro with the full thing, thanks. It's because the inside of the <<<JS is supposed to be colorized as javascript, and something is going wrong so that it colorizes the whole file as JS.

[lewisconklin]

Hello, I am new to the community and would like to work on this issue. I am able to reproduce the bug locally. Would I be able to become an assignee? Thank you!

[roblourens]

@lewisconklin, thanks, this is probably not an easy first issue though

[lewisconklin]

@roblourens Thanks. I believe you are correct, as it seems I have hit a wall in my progress with this issue. Here is what I have so far:

1. In PHP, inside of double-quoted strings and heredocs, {} must be used to surround complex variables. In the above example, they are needed due to the array with quoted index.

2. In the above example, using the scope inspector, it is clear that TextMate does not recognize {$_SESSION['ans_draft_post']} as a PHP variable. This is what breaks the syntax highlighting.

3. Using a heredoc with a language such as SQL, where {} do not naturally occur, this bug does not appear.

4. This bug does not occur within a JavaScript heredoc unless it is used inside a function call, as a variable assignment, etc. I have provided the following, simpler example to demonstrate this.

<?php

$arr = array('foo' => 'bar');

$test=<<<JS
    {$arr['foo']};  // works fine
    let bar = {$arr['foo']};  // breaks syntax highlighting
JS;

print "syntax highlighting test";

?>

I believe the reason for this is due to TextMate's scope selecting rules. In the php.tmLanguage.json file, under the heredoc section for JavaScript, we have

"name": "meta.embedded.js",
"patterns": [
    {
        "include": "#interpolation"
    },
    {
        "include": "source.js"
    }
]

When a {$} is encountered in a JavaScript heredoc, a match is found using both the #interpolation and the source.js grammar rules. From the TextMate docs:

If more than one scope selector matches the current scope then they are ranked 
according to how “good” a match they each are.

The winner is the scope selector which (in order of precedence):

1. Match the element deepest down in the scope e.g. string wins over source.php 
when the scope is source.php string.quoted.

2. Match most of the deepest element e.g. string.quoted wins over string.

In my provided example, the first case works because both the interpolated PHP and JavaScript are on a equal level of scope, and I believe that since #interpolation is included first in the patterns array, it has precedence. In the second case, the JavaScript scope is now deeper due to the variable assignment, and thus wins over the PHP grammar. Perhaps this is a case where we need scope exclusion?

[roblourens]

Thanks for the details. Want to warn you that we are currently behind the latest work upstream, and I don't know whether any other changes have been made that would impact this. If you only looked at the grammar in our repo, then if you're going to spend more time on it you may want to check that. Some info here https://github.com/microsoft/vscode/pull/111922#issuecomment-739609520

[KapitanOczywisty]

@lewisconklin You're more or less correct until this sentence:

When a {$} is encountered in a JavaScript heredoc, a match is found using both the #interpolation and the source.js grammar rules. From the TextMate docs:

When TM is in any syntax block it is checking only the rules defined or included via include: #whatever in that exact block. You can reference $self, but this would only include the top-level rules from the source.js. There is no practical way to include the #interpolation rules when there are blocks involved. On the other hand, the SQL have a very limited amount of blocks, so you could just replace the string definition with your own, which is exactly what is done, but this also is not perfect and you can find many issues related to broken SQL syntax in language-php.

---

I'd strongly advice you to avoid injections all together, for the SQL you can use a prepared statements, for the JS/CSS I'd send a single JSON object with necessary data for a separate JS to deal with, and for HTML use a template system of some sort. There is no real point in mixing languages these days.

[PrinceOfAbyss]

In a piece of code I have a PHP Heredoc where I've included some JS code, with the appropriate JS mark/identifier, so that the engine gets to know that following code will be JS. In the Heredoc I needed to include some PHP variables. The problem is that the formatter fails to identify the PHP vars inside the Heredoc correctly; I tried either with curly braces around the vars, in which case the formatter didn't even recognize the closing Heredoc syntax, failing even to format the rest of the PHP code after the Heredoc. I then tried without curly braces around the PHP vars, in which case formatting came back, but this time it detected (erroneously) errors regarding the vars.

An extract from my code where you can see all the above happening is the following:

<?php
if (!isset($wp_query->query_vars['publication'])) {
// other commands

    echo <<<JS
<script>
$(function() {
    $('#date-filter').datepicker({
        dateFormat: 'dd/mm/yy',
        minDate: new Date($min['y'], $min['m'], $min['d']),
        maxDate: new Date($max['y'], $max['m'], $max['d']),
        showButtonPanel: true,
        onSelect: function(dateText) {
            if (dateText == ddmmyy(new Date())) {
                window.location.href = location.protocol + '//' + location.host + '/protoselida/';
            } else {
                window.location.href = location.protocol + '//' + location.host + '/protoselida/' + yymmdd(dateText);
            }
        }
    }).datepicker('setDate', new Date($cur['y'], $cur['m'], $cur['d']));

    $('#frontpages-genre-filter span a').hover(function() {
        $(this).addClass('ui-state-hover');
    }, function() {
        $(this).removeClass('ui-state-hover');
    }).click(function() {
        $(this).parent().siblings().children('a').removeClass('ui-state-active');
        $(this).addClass('ui-state-active');
    });
});

$(document).ready(function() {
    $('#frontpages-genre-filter span a').each(function(index) {
        if ($(this).data('hash') == window.location.hash.substr(1)) {
            $(this).addClass('ui-state-active');
        }
    });
});
</script>
JS;
} else {
    $date = date('d-m-Y', strtotime(intval($wp_query->query_vars['date'])));
    $publication = sanitize_title($wp_query->query_vars['publication']);
}

Funny thing is that out of the 9 instances where I use the curly braces in the Heredoc, it only gets confused with the 3 ones on line 288. See in the video I recorded below, how the color highlighting gets back as soon as I remove the curly braces from line 288, while there are still 6 more on lines 278 and 279.

https://youtu.be/oSatAM5WfsU

Some relevant screenshots:

Platform: Windows 10 Pro 20H2 IDE: VS Code v1.55.2

[KapitanOczywisty]

@PrinceOfAbyss Hi, in your case, the <script> tag is messing everything. In JavaScript it's treated as JSX, but this is not correct JSX format, thus colors are getting weird. Remove <script> from JS or change heredoc type to HTML.

Also without braces highlighting will be better, since it's treated as somewhat valid JS code.

[PrinceOfAbyss]

@KapitanOczywisty I tried it with HTML identifier, same thing, just different colors, as it was 'signaled' as being HTML... The only difference was achieved using something generic, like CODE, where the heredoc was highlighted in olive green, andbthe rest of the PHP code (after line 265 that is) was correctly highlighted in PHP color scheme. But that's justva workaround.

There's definitely a bug here..

[KapitanOczywisty]

I tried it with HTML identifier

and without braces for interpolation? Try also JS without <script> tag.

[PrinceOfAbyss]

Yup, without braces also. I was getting that T_ENCAPSED error again... I haven't tried it without ?

[KapitanOczywisty]

@PrinceOfAbyss I mean to extract tag outside.

echo "<script>";
echo <<<JS
...
JS;
echo "</script>";

About interpolation, without braces you should remove quotes from indexes e.g. $min[x] etc 3v4l docs

The best would be to pass JSON to HTML, and keep JS code separate from PHP.

$info = htmlspecialchars(json_encode(['min'=>$min, 'max'=>$max, 'cur'=>$cur]), ENT_QUOTES, 'UTF-8');
echo "<script src=\"some-script.js\" data-info=\"{$info}\" />";

and some-script.js

const {min, max, cur} = $(document.currentScript).data('info');
// ...

[PrinceOfAbyss]

Ah, that's what you meant! Sorry I didn't get it before! Well, I'll try both and will report back! Thanks a lot for your help!

[PrinceOfAbyss]

@PrinceOfAbyss I mean to extract tag outside.

echo "<script>";
echo <<<JS
...
JS;
echo "</script>";

About interpolation, without braces you should remove quotes from indexes e.g. $min[x] etc 3v4l docs

The best would be to pass JSON to HTML, and keep JS code separate from PHP.

$info = htmlspecialchars(json_encode(['min'=>$min, 'max'=>$max, 'cur'=>$cur]), ENT_QUOTES, 'UTF-8');
echo "<script src=\"some-script.js\" data-info=\"{$info}\" />";

and some-script.js

const {min, max, cur} = $(document.currentScript).data('info');
// ...

I owe an update on this. So here it is: Indeed removing the curly braces and the quotes from the indices corrected and highlighting issue. Now everything works and looks great.

And as a bonus: This JS formatter/beautifier formats my code just fine. With the curly braces and quotes in place it was getting confused...

So thanks a LOT @KapitanOczywisty for your help! It was really priceless!

    $('#date-filter').datepicker({
        dateFormat: 'dd/mm/yy',
        minDate: new Date($min[y], $min[m], $min[d]),
        maxDate: new Date($max[y], $max[m], $max[d]),
        showButtonPanel: true,
        onSelect: function(dateText) {
            if (dateText == ddmmyy(new Date())) {
                window.location.href = location.protocol + '//' + location.host + '/protoselida/';
            } else {
                window.location.href = location.protocol + '//' + location.host + '/protoselida/' + yymmdd(dateText);
            }
        }
    }).datepicker('setDate', new Date($cur[y], $cur[m], $cur[d]));

    $('#frontpages-genre-filter span a').hover(function() {
        $(this).addClass('ui-state-hover');
    }, function() {
        $(this).removeClass('ui-state-hover');
    }).click(function() {
        $(this).parent().siblings().children('a').removeClass('ui-state-active');
        $(this).addClass('ui-state-active');
    });