search

universal-ctags / ctags

A maintained ctags implementation
https://ctags.io
GNU General Public License v2.0
6.59k stars 629 forks source link

Java?: duplicated list of results #1830

Open wmm125 opened 6 years ago

wmm125 commented 6 years ago

I'm running universal ctags in interactive on a java code (also available at: https://raw.githubusercontent.com/ShadeWalker/Tango_AL813/master/packages/apps/Email/src/com/android/email/activity/setup/AccountSetupNoteDialogFragment.java) and I got 36 json output tags when expected result is only 16. This behavior can not be reproduced in non interactive mode.

The name of the parser:

The command line you used to run ctags:

$ bin/ctags --options=NONE --_interactive --guess-language-eagerly --c-kinds=+l --java-kinds=+l --sql-kinds=+l --Fortran-kinds=+L --C++-kinds=+l --file-scope=yes -u --fields=-anf+iKnS --excmd=pattern --langmap=sh:+.kshlib --langmap=sql:+.plb --langmap=sql:+.pls --langmap=sql:+.pld --langmap=sql:+.pks

The content of input file:

/*
* Copyright (C) 2014 MediaTek Inc.
* Modification based on code covered by the mentioned copyright
* and/or permission notice(s).
*/
/*
 * Copyright (C) 2014 The Android Open Source Project
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package com.android.email.activity.setup;

import android.app.AlertDialog;
import android.app.Dialog;
import android.app.DialogFragment;
import android.content.Context;
import android.content.DialogInterface;
import android.os.Bundle;

import com.android.email.R;

public class AccountSetupNoteDialogFragment extends DialogFragment {
    public static final String TAG = "NoteDialogFragment";

    // Argument bundle keys
    private static final String BUNDLE_KEY_NOTE = "NoteDialogFragment.Note";

    public static interface Callback {
        void onNoteDialogComplete();
        void onNoteDialogCancel();
    }

    // Public no-args constructor needed for fragment re-instantiation
    public AccountSetupNoteDialogFragment() {}

    /**
     * Create the dialog with parameters
     */
    public static AccountSetupNoteDialogFragment newInstance(String note) {
        final AccountSetupNoteDialogFragment f = new AccountSetupNoteDialogFragment();
        final Bundle b = new Bundle(1);
        b.putString(BUNDLE_KEY_NOTE, note);
        f.setArguments(b);
        return f;
    }

    @Override
    public Dialog onCreateDialog(Bundle savedInstanceState) {
        final Context context = getActivity();
        final String note = getArguments().getString(BUNDLE_KEY_NOTE);

        setCancelable(true);

        return new AlertDialog.Builder(context)
                .setIconAttribute(android.R.attr.alertDialogIcon)
                .setTitle(android.R.string.dialog_alert_title)
                .setMessage(note)
                .setPositiveButton(
                        android.R.string.ok,
                        new DialogInterface.OnClickListener() {
                            @Override
                            public void onClick(DialogInterface dialog, int which) {
                                final Callback a = (Callback) getActivity();
                                a.onNoteDialogComplete();
                                dismiss();
                            }
                        })
                .setNegativeButton(
                        context.getString(android.R.string.cancel),
                        new DialogInterface.OnClickListener() {
                            @Override
                            public void onClick(DialogInterface dialog, int which) {
                                dialog.cancel();
                            }
                        })
                .create();
    }

    @Override
    public void onCancel(DialogInterface dialog) {
        super.onCancel(dialog);
        final Callback a = (Callback) getActivity();
        a.onNoteDialogCancel();
    }
}

Interactive mode header:

{"size": 3410, "command": "generate-tags", "filename": "AccountSetupNoteDialogFragment.java"}

The tags output you are not satisfied with:

{"_type": "tag", "name": "com.android.email.activity.setup", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^package com.android.email.activity.setup;$/", "line": 22, "kind": "package"}
{"_type": "tag", "name": "AccountSetupNoteDialogFragment", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^public class AccountSetupNoteDialogFragment extends DialogFragment {$/", "inherits": "DialogFragment", "line": 33, "kind": "class"}
{"_type": "tag", "name": "TAG", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public static final String TAG = \"NoteDialogFragment\";$/", "line": 34, "kind": "field", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "BUNDLE_KEY_NOTE", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    private static final String BUNDLE_KEY_NOTE = \"NoteDialogFragment.Note\";$/", "line": 37, "kind": "field", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "Callback", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public static interface Callback {$/", "line": 39, "kind": "interface", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "onNoteDialogComplete", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        void onNoteDialogComplete();$/", "line": 40, "signature": "()", "kind": "method", "scope": "AccountSetupNoteDialogFragment.Callback", "scopeKind": "interface"}
{"_type": "tag", "name": "onNoteDialogCancel", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        void onNoteDialogCancel();$/", "line": 41, "signature": "()", "kind": "method", "scope": "AccountSetupNoteDialogFragment.Callback", "scopeKind": "interface"}
{"_type": "tag", "name": "AccountSetupNoteDialogFragment", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public AccountSetupNoteDialogFragment() {}$/", "line": 45, "signature": "()", "kind": "method", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "newInstance", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public static AccountSetupNoteDialogFragment newInstance(String note) {$/", "line": 50, "signature": "(String note)", "kind": "method", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "f", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        final AccountSetupNoteDialogFragment f = new AccountSetupNoteDialogFragment();$/", "line": 51, "kind": "local"}
{"_type": "tag", "name": "b", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        final Bundle b = new Bundle(1);$/", "line": 52, "kind": "local"}
{"_type": "tag", "name": "onCreateDialog", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public Dialog onCreateDialog(Bundle savedInstanceState) {$/", "line": 59, "signature": "(Bundle savedInstanceState)", "kind": "method", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "context", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        final Context context = getActivity();$/", "line": 60, "kind": "local"}
{"_type": "tag", "name": "note", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        final String note = getArguments().getString(BUNDLE_KEY_NOTE);$/", "line": 61, "kind": "local"}
{"_type": "tag", "name": "context.getString", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^                        context.getString(android.R.string.cancel),$/", "line": 80, "signature": "(android.R.string.cancel)", "kind": "method"}
{"_type": "tag", "name": "DialogInterface.OnClickListener", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^                        new DialogInterface.OnClickListener() {$/", "line": 81, "signature": "()", "kind": "method"}
{"_type": "tag", "name": "onClick", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^                            public void onClick(DialogInterface dialog, int which) {$/", "line": 83, "signature": "(DialogInterface dialog, int which)", "kind": "method"}
{"_type": "tag", "name": "create", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^                .create();$/", "line": 87, "signature": "()", "kind": "method"}
{"_type": "tag", "name": "com.android.email.activity.setup", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^package com.android.email.activity.setup;$/", "line": 22, "kind": "package"}
{"_type": "tag", "name": "AccountSetupNoteDialogFragment", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^public class AccountSetupNoteDialogFragment extends DialogFragment {$/", "inherits": "DialogFragment", "line": 33, "kind": "class"}
{"_type": "tag", "name": "TAG", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public static final String TAG = \"NoteDialogFragment\";$/", "line": 34, "kind": "field", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "BUNDLE_KEY_NOTE", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    private static final String BUNDLE_KEY_NOTE = \"NoteDialogFragment.Note\";$/", "line": 37, "kind": "field", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "Callback", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public static interface Callback {$/", "line": 39, "kind": "interface", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "onNoteDialogComplete", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        void onNoteDialogComplete();$/", "line": 40, "signature": "()", "kind": "method", "scope": "AccountSetupNoteDialogFragment.Callback", "scopeKind": "interface"}
{"_type": "tag", "name": "onNoteDialogCancel", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        void onNoteDialogCancel();$/", "line": 41, "signature": "()", "kind": "method", "scope": "AccountSetupNoteDialogFragment.Callback", "scopeKind": "interface"}
{"_type": "tag", "name": "AccountSetupNoteDialogFragment", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public AccountSetupNoteDialogFragment() {}$/", "line": 45, "signature": "()", "kind": "method", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "newInstance", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public static AccountSetupNoteDialogFragment newInstance(String note) {$/", "line": 50, "signature": "(String note)", "kind": "method", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "f", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        final AccountSetupNoteDialogFragment f = new AccountSetupNoteDialogFragment();$/", "line": 51, "kind": "local"}
{"_type": "tag", "name": "b", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        final Bundle b = new Bundle(1);$/", "line": 52, "kind": "local"}
{"_type": "tag", "name": "onCreateDialog", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public Dialog onCreateDialog(Bundle savedInstanceState) {$/", "line": 59, "signature": "(Bundle savedInstanceState)", "kind": "method", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "context", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        final Context context = getActivity();$/", "line": 60, "kind": "local"}
{"_type": "tag", "name": "note", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        final String note = getArguments().getString(BUNDLE_KEY_NOTE);$/", "line": 61, "kind": "local"}
{"_type": "tag", "name": "context.getString", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^                        context.getString(android.R.string.cancel),$/", "line": 80, "signature": "(android.R.string.cancel)", "kind": "method"}
{"_type": "tag", "name": "DialogInterface.OnClickListener", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^                        new DialogInterface.OnClickListener() {$/", "line": 81, "signature": "()", "kind": "method"}
{"_type": "tag", "name": "onClick", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^                            public void onClick(DialogInterface dialog, int which) {$/", "line": 83, "signature": "(DialogInterface dialog, int which)", "kind": "method"}
{"_type": "tag", "name": "create", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^                .create();$/", "line": 87, "signature": "()", "kind": "method"}
{"_type": "completed", "command": "generate-tags"}

The tags output you expect:

{"_type": "tag", "name": "com.android.email.activity.setup", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^package com.android.email.activity.setup;$/", "line": 22, "kind": "package"}
{"_type": "tag", "name": "AccountSetupNoteDialogFragment", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^public class AccountSetupNoteDialogFragment extends DialogFragment {$/", "inherits": "DialogFragment", "line": 33, "kind": "class"}
{"_type": "tag", "name": "TAG", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public static final String TAG = \"NoteDialogFragment\";$/", "line": 34, "kind": "field", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "BUNDLE_KEY_NOTE", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    private static final String BUNDLE_KEY_NOTE = \"NoteDialogFragment.Note\";$/", "line": 37, "kind": "field", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "Callback", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public static interface Callback {$/", "line": 39, "kind": "interface", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "onNoteDialogComplete", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        void onNoteDialogComplete();$/", "line": 40, "signature": "()", "kind": "method", "scope": "AccountSetupNoteDialogFragment.Callback", "scopeKind": "interface"}
{"_type": "tag", "name": "onNoteDialogCancel", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        void onNoteDialogCancel();$/", "line": 41, "signature": "()", "kind": "method", "scope": "AccountSetupNoteDialogFragment.Callback", "scopeKind": "interface"}
{"_type": "tag", "name": "AccountSetupNoteDialogFragment", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public AccountSetupNoteDialogFragment() {}$/", "line": 45, "signature": "()", "kind": "method", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "newInstance", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public static AccountSetupNoteDialogFragment newInstance(String note) {$/", "line": 50, "signature": "(String note)", "kind": "method", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "f", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        final AccountSetupNoteDialogFragment f = new AccountSetupNoteDialogFragment();$/", "line": 51, "kind": "local"}
{"_type": "tag", "name": "b", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        final Bundle b = new Bundle(1);$/", "line": 52, "kind": "local"}
{"_type": "tag", "name": "onCreateDialog", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^    public Dialog onCreateDialog(Bundle savedInstanceState) {$/", "line": 59, "signature": "(Bundle savedInstanceState)", "kind": "method", "scope": "AccountSetupNoteDialogFragment", "scopeKind": "class"}
{"_type": "tag", "name": "context", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        final Context context = getActivity();$/", "line": 60, "kind": "local"}
{"_type": "tag", "name": "note", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^        final String note = getArguments().getString(BUNDLE_KEY_NOTE);$/", "line": 61, "kind": "local"}
{"_type": "tag", "name": "context.getString", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^                        context.getString(android.R.string.cancel),$/", "line": 80, "signature": "(android.R.string.cancel)", "kind": "method"}
{"_type": "tag", "name": "DialogInterface.OnClickListener", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^                        new DialogInterface.OnClickListener() {$/", "line": 81, "signature": "()", "kind": "method"}
{"_type": "tag", "name": "onClick", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^                            public void onClick(DialogInterface dialog, int which) {$/", "line": 83, "signature": "(DialogInterface dialog, int which)", "kind": "method"}
{"_type": "tag", "name": "create", "path": "AccountSetupNoteDialogFragment.java", "pattern": "/^                .create();$/", "line": 87, "signature": "()", "kind": "method"}
{"_type": "completed", "command": "generate-tags"}

The version of ctags:

$ ctags --version
Universal Ctags 0.0.0(0645b2c), Copyright (C) 2015 Universal Ctags Team
Universal Ctags is derived from Exuberant Ctags.
Exuberant Ctags 5.8, Copyright (C) 1996-2009 Darren Hiebert
  Compiled: Jul  4 2017, 16:12:00
  URL: https://ctags.io/
  Optional compiled features: +wildcards, +regex, +multibyte, +option-directory, +json, +interactive

How do you get ctags binary:

Building it locally with no code change, only enabling json output

masatake commented 6 years ago

Reproduced with the following shorten input:

public class AccountSetupNoteDialogFragment {
    public Dialog onCreateDialog(Bundle savedInstanceState) {
        return new AlertDialog.Builder(context)
                .setPositiveButton(
                        new DialogInterface.OnClickListener() {
                            public void onClick(DialogInterface dialog, int which) {
                                final Callback a = (Callback) getActivity();
                            }
                        })
    }
}

The output:

[jet@localhost]~/var/ctags%  u-ctags --options=NONE --java-kinds=+l --file-scope=yes -u --fields=-anfs+n  -o - /tmp/foo.java
 u-ctags --options=NONE --java-kinds=+l --file-scope=yes -u --fields=-anfs+n  -o - /tmp/foo.java
u-ctags: Notice: No options will be read from files or environment
AccountSetupNoteDialogFragment  /tmp/foo.java   /^public class AccountSetupNoteDialogFragment {$/;" c   line:1
onCreateDialog  /tmp/foo.java   /^    public Dialog onCreateDialog(Bundle savedInstanceState) {$/;" m   line:2
AccountSetupNoteDialogFragment  /tmp/foo.java   /^public class AccountSetupNoteDialogFragment {$/;" c   line:1
onCreateDialog  /tmp/foo.java   /^    public Dialog onCreateDialog(Bundle savedInstanceState) {$/;" m   line:2

If the lines inside onCall method empty, it is not reproduced:

[jet@localhost]~/var/ctags% cat /tmp/foo.java 
cat /tmp/foo.java 
public class AccountSetupNoteDialogFragment {
    public Dialog onCreateDialog(Bundle savedInstanceState) {
        return new AlertDialog.Builder(context)
                .setPositiveButton(
                        new DialogInterface.OnClickListener() {
                            public void onClick(DialogInterface dialog, int which) {
                            }
                        })
    }
}
[jet@localhost]~/var/ctags%  u-ctags --options=NONE --java-kinds=+l --file-scope=yes -u --fields=-anfs+n  -o - /tmp/foo.java
 u-ctags --options=NONE --java-kinds=+l --file-scope=yes -u --fields=-anfs+n  -o - /tmp/foo.java
u-ctags: Notice: No options will be read from files or environment
AccountSetupNoteDialogFragment  /tmp/foo.java   /^public class AccountSetupNoteDialogFragment {$/;" c   line:1
onCreateDialog  /tmp/foo.java   /^    public Dialog onCreateDialog(Bundle savedInstanceState) {$/;" m   line:2

So I guess this is a bug of Java parser.

masatake commented 6 years ago

This is not reproduced in Exuberant-ctags.

masatake commented 6 years ago

The shorten input I made misses a semicolon. It should be:

public class AccountSetupNoteDialogFragment {
    public Dialog onCreateDialog(Bundle savedInstanceState) {
        return new AlertDialog.Builder(context)
                .setPositiveButton(
                        new DialogInterface.OnClickListener() {
                            public void onClick(DialogInterface dialog, int which) {
                                final Callback a = (Callback) getActivity();
                            }
                        });
    }
}

For the above code, new Java parser I tested in #1847 emits following tags:

AccountSetupNoteDialogFragment  baz.java    /^public class AccountSetupNoteDialogFragment {$/;" c   line:1
onCreateDialog  baz.java    /^    public Dialog onCreateDialog(Bundle savedInstanceState) {$/;" m   line:2
__anon37c95d920201  baz.java    /^                        new DialogInterface.OnClickListener() {$/;"   c   line:5
onClick baz.java    /^                            public void onClick(DialogInterface dialog, int which) {$/;"  m   line:6
a   baz.java    /^                                final Callback a = (Callback) getActivity();$/;"  l   line:7

The new parser doesn't emit the duplicated tags. It takes more time to merge the experimental code. But it solves the issue you reported. If you are interested in trying the code, let me know.