linq2db / linq2db.LINQPad

linq2db.LINQPad is a driver for LINQPad.
MIT License
74 stars 23 forks source link

NRE in OracleSchema provider #72

Closed MaceWindu closed 1 year ago

MaceWindu commented 2 years ago

https://forum.linqpad.net/discussion/2741/linq-to-db-oracle-connection-successful-but-population-fails

image

Would be hard to track it down. Looks like stack trace is optimized: GetSchema cannot throw this exception and I don't see currently how it could fail in called methods.

KevinBurton commented 2 years ago

I am not sure how the conclusion that 'GetSchema cannot throw this exception' was reached. This method is huge

        public virtual DatabaseSchema GetSchema(DataConnection dataConnection, GetSchemaOptions? options = null)
        {
            GetSchemaOptions options2 = options;
            if (options2 == null)
            {
                options2 = new GetSchemaOptions();
            }

            IncludedSchemas = GetHashSet(options2.IncludedSchemas, options2.StringComparer);
            ExcludedSchemas = GetHashSet(options2.ExcludedSchemas, options2.StringComparer);
            IncludedCatalogs = GetHashSet(options2.IncludedCatalogs, options2.StringComparer);
            ExcludedCatalogs = GetHashSet(options2.ExcludedCatalogs, options2.StringComparer);
            GenerateChar1AsString = options2.GenerateChar1AsString;
            DbConnection connection = dataConnection.Connection;
            InitProvider(dataConnection);
            DataTypesDic = new Dictionary<string, DataTypeInfo>(StringComparer.OrdinalIgnoreCase);
            ProviderSpecificDataTypesDic = new Dictionary<string, DataTypeInfo>(StringComparer.OrdinalIgnoreCase);
            DataTypesByProviderDbTypeDic = new Dictionary<int, DataTypeInfo>();
            ProviderSpecificDataTypesByProviderDbTypeDic = new Dictionary<int, DataTypeInfo>();
            foreach (DataTypeInfo dataType2 in GetDataTypes(dataConnection))
            {
                if (dataType2.ProviderSpecific)
                {
                    if (!ProviderSpecificDataTypesDic.ContainsKey(dataType2.TypeName))
                    {
                        ProviderSpecificDataTypesDic.Add(dataType2.TypeName, dataType2);
                    }

                    if (!ProviderSpecificDataTypesByProviderDbTypeDic.ContainsKey(dataType2.ProviderDbType))
                    {
                        ProviderSpecificDataTypesByProviderDbTypeDic.Add(dataType2.ProviderDbType, dataType2);
                    }
                }
                else
                {
                    if (!DataTypesDic.ContainsKey(dataType2.TypeName))
                    {
                        DataTypesDic.Add(dataType2.TypeName, dataType2);
                    }

                    if (!DataTypesByProviderDbTypeDic.ContainsKey(dataType2.ProviderDbType))
                    {
                        DataTypesByProviderDbTypeDic.Add(dataType2.ProviderDbType, dataType2);
                    }
                }
            }

            List<TableSchema> list;
            if (options2.GetTables)
            {
                list = (from t in GetTables(dataConnection, options2)
                        where (IncludedSchemas.Count == 0 || IncludedSchemas.Contains(t.SchemaName)) && (ExcludedSchemas.Count == 0 || !ExcludedSchemas.Contains(t.SchemaName)) && (IncludedCatalogs.Count == 0 || IncludedCatalogs.Contains(t.CatalogName)) && (ExcludedCatalogs.Count == 0 || !ExcludedCatalogs.Contains(t.CatalogName)) && (options2.LoadTable == null || options2.LoadTable!(new LoadTableData(t)))
                        select new TableSchema
                        {
                            ID = t.TableID,
                            CatalogName = t.CatalogName,
                            SchemaName = t.SchemaName,
                            TableName = t.TableName,
                            Description = t.Description,
                            IsDefaultSchema = t.IsDefaultSchema,
                            IsView = t.IsView,
                            TypeName = ToValidName(t.TableName),
                            Columns = new List<ColumnSchema>(),
                            ForeignKeys = new List<ForeignKeySchema>(),
                            IsProviderSpecific = t.IsProviderSpecific
                        }).ToList();
                IReadOnlyCollection<PrimaryKeyInfo> primaryKeys = GetPrimaryKeys(dataConnection, list, options2);
                var enumerable = from _003C_003Eh__TransparentIdentifier2 in Enumerable.Join(from c in GetColumns(dataConnection, options2)
                                                                                             join pk in primaryKeys on c.TableID + "." + c.Name equals pk.TableID + "." + pk.ColumnName into g2
                                                                                             from pk in g2.DefaultIfEmpty()
                                                                                             select new { _003C_003Eh__TransparentIdentifier0, pk }, list, _003C_003Eh__TransparentIdentifier1 => _003C_003Eh__TransparentIdentifier1._003C_003Eh__TransparentIdentifier0.c.TableID, (TableSchema t) => t.ID, (_003C_003Eh__TransparentIdentifier1, TableSchema t) => new { _003C_003Eh__TransparentIdentifier1, t })
                                 orderby _003C_003Eh__TransparentIdentifier2._003C_003Eh__TransparentIdentifier1._003C_003Eh__TransparentIdentifier0.c.Ordinal
                                 select new
                                 {
                                     t = _003C_003Eh__TransparentIdentifier2.t,
                                     c = _003C_003Eh__TransparentIdentifier2._003C_003Eh__TransparentIdentifier1._003C_003Eh__TransparentIdentifier0.c,
                                     dt = GetDataType(_003C_003Eh__TransparentIdentifier2._003C_003Eh__TransparentIdentifier1._003C_003Eh__TransparentIdentifier0.c.DataType, _003C_003Eh__TransparentIdentifier2._003C_003Eh__TransparentIdentifier1._003C_003Eh__TransparentIdentifier0.c.Type, options2),
                                     pk = _003C_003Eh__TransparentIdentifier2._003C_003Eh__TransparentIdentifier1.pk
                                 };
                foreach (var item in enumerable)
                {
                    string dataType = item.c.DataType;
                    Type systemType = GetSystemType(dataType, item.c.ColumnType, item.dt, item.c.Length, item.c.Precision, item.c.Scale, options2);
                    bool isNullable = item.c.IsNullable;
                    string columnType = item.c.ColumnType ?? GetDbType(options2, dataType, item.dt, item.c.Length, item.c.Precision, item.c.Scale, null, null, null);
                    item.t.Columns.Add(new ColumnSchema
                    {
                        Table = item.t,
                        ColumnName = item.c.Name,
                        ColumnType = columnType,
                        IsNullable = isNullable,
                        MemberName = ToValidName(item.c.Name),
                        MemberType = ToTypeName(systemType, isNullable),
                        SystemType = systemType,
                        DataType = (item.c.Type ?? GetDataType(dataType, item.c.ColumnType, item.c.Length, item.c.Precision, item.c.Scale)),
                        ProviderSpecificType = GetProviderSpecificType(dataType),
                        SkipOnInsert = (item.c.SkipOnInsert || item.c.IsIdentity),
                        SkipOnUpdate = (item.c.SkipOnUpdate || item.c.IsIdentity),
                        IsPrimaryKey = (item.pk != null),
                        PrimaryKeyOrder = (item.pk?.Ordinal ?? (-1)),
                        IsIdentity = item.c.IsIdentity,
                        Description = item.c.Description,
                        Length = item.c.Length,
                        Precision = item.c.Precision,
                        Scale = item.c.Scale
                    });
                }

                IReadOnlyCollection<ForeignKeyInfo> readOnlyCollection;
                if (!options2.GetForeignKeys)
                {
                    IReadOnlyCollection<ForeignKeyInfo> empty = (IReadOnlyCollection<ForeignKeyInfo>)(object)Array<ForeignKeyInfo>.Empty;
                    readOnlyCollection = empty;
                }
                else
                {
                    readOnlyCollection = GetForeignKeys(dataConnection, list, options2);
                }

                IReadOnlyCollection<ForeignKeyInfo> source = readOnlyCollection;
                foreach (ForeignKeyInfo fk in source.OrderBy((ForeignKeyInfo f) => f.Ordinal))
                {
                    TableSchema tableSchema = list.Where((TableSchema t) => t.ID == fk.ThisTableID).FirstOrDefault();
                    TableSchema tableSchema2 = list.Where((TableSchema t) => t.ID == fk.OtherTableID).FirstOrDefault();
                    if (tableSchema == null || tableSchema2 == null)
                    {
                        continue;
                    }

                    StringComparison stringComparison = ForeignKeyColumnComparison(fk.OtherColumn);
                    ColumnSchema columnSchema = tableSchema.Columns.Where((ColumnSchema c) => c.ColumnName == fk.ThisColumn).SingleOrDefault();
                    ColumnSchema columnSchema2 = tableSchema2.Columns.Where((ColumnSchema c) => string.Equals(c.ColumnName, fk.OtherColumn, stringComparison)).SingleOrDefault();
                    if (columnSchema != null && columnSchema2 != null)
                    {
                        ForeignKeySchema foreignKeySchema = tableSchema.ForeignKeys.FirstOrDefault((ForeignKeySchema f) => f.KeyName == fk.Name);
                        if (foreignKeySchema == null)
                        {
                            foreignKeySchema = new ForeignKeySchema
                            {
                                KeyName = fk.Name,
                                MemberName = ToValidName(fk.Name),
                                ThisTable = tableSchema,
                                OtherTable = tableSchema2,
                                ThisColumns = new List<ColumnSchema>(),
                                OtherColumns = new List<ColumnSchema>(),
                                CanBeNull = true
                            };
                            tableSchema.ForeignKeys.Add(foreignKeySchema);
                        }

                        foreignKeySchema.ThisColumns.Add(columnSchema);
                        foreignKeySchema.OtherColumns.Add(columnSchema2);
                    }
                }

                List<TableSchema> providerSpecificTables = GetProviderSpecificTables(dataConnection, options2);
                if (providerSpecificTables != null)
                {
                    list.AddRange(providerSpecificTables);
                }
            }
            else
            {
                list = new List<TableSchema>();
            }

            List<ProcedureSchema> list2;
            if (options2.GetProcedures)
            {
                ISqlBuilder sqlBuilder = dataConnection.DataProvider.CreateSqlBuilder(dataConnection.MappingSchema);
                List<ProcedureInfo> procedures = GetProcedures(dataConnection, options2);
                int i = 0;
                if (procedures != null)
                {
                    IEnumerable<ProcedureParameterInfo> procedureParameters = GetProcedureParameters(dataConnection, procedures, options2);
                    IEnumerable<ProcedureParameterInfo> inner = procedureParameters ?? Array<ProcedureParameterInfo>.Empty;
                    list2 = (from sp in procedures
                             where (IncludedSchemas.Count == 0 || IncludedSchemas.Contains(sp.SchemaName)) && (ExcludedSchemas.Count == 0 || !ExcludedSchemas.Contains(sp.SchemaName)) && (IncludedCatalogs.Count == 0 || IncludedCatalogs.Contains(sp.CatalogName)) && (ExcludedCatalogs.Count == 0 || !ExcludedCatalogs.Contains(sp.CatalogName))
                             join p in inner on sp.ProcedureID equals p.ProcedureID into gr
                             select new ProcedureSchema
                             {
                                 CatalogName = sp.CatalogName,
                                 SchemaName = sp.SchemaName,
                                 PackageName = sp.PackageName,
                                 ProcedureName = sp.ProcedureName,
                                 MemberName = ToValidName(sp.PackageName + ((sp.PackageName != null) ? "_" : null) + sp.ProcedureName),
                                 IsFunction = sp.IsFunction,
                                 IsTableFunction = sp.IsTableFunction,
                                 IsResultDynamic = sp.IsResultDynamic,
                                 IsAggregateFunction = sp.IsAggregateFunction,
                                 IsDefaultSchema = sp.IsDefaultSchema,
                                 Description = sp.Description,
                                 Parameters = (from pr in gr
                                               let dt = GetDataType(pr.DataType, null, options2)
                                               let systemType = GetSystemType(pr.DataType, pr.DataTypeExact, dt, pr.Length, pr.Precision, pr.Scale, options2)
                                               orderby pr.Ordinal
                                               select new ParameterSchema
                                               {
                                                   SchemaName = pr.ParameterName,
                                                   SchemaType = GetDbType(options2, pr.DataType, dt, pr.Length, pr.Precision, pr.Scale, pr.UDTCatalog, pr.UDTSchema, pr.UDTName),
                                                   IsIn = pr.IsIn,
                                                   IsOut = pr.IsOut,
                                                   IsResult = pr.IsResult,
                                                   Size = pr.Length,
                                                   ParameterName = ToValidName(pr.ParameterName ?? ("par" + ++i)),
                                                   ParameterType = ToTypeName(systemType, isNullable: true),
                                                   SystemType = systemType,
                                                   DataType = GetDataType(pr.DataType, pr.DataTypeExact, pr.Length, pr.Precision, pr.Scale),
                                                   ProviderSpecificType = GetProviderSpecificType(pr.DataType),
                                                   IsNullable = pr.IsNullable,
                                                   Description = pr.Description
                                               }).ToList()
                             } into ps
                             select (ps)).ToList();
                    int num = 1;
                    bool flag = dataConnection.Transaction != null;
                    if (GetProcedureSchemaExecutesProcedure && flag)
                    {
                        throw new LinqToDBException("Cannot read schema with GetSchemaOptions.GetProcedures = true from transaction. Remove transaction or set GetSchemaOptions.GetProcedures to false");
                    }

                    if (!flag)
                    {
                        dataConnection.BeginTransaction();
                    }

                    try
                    {
                        foreach (ProcedureSchema item2 in list2)
                        {
                            if (!item2.IsResultDynamic && (!item2.IsFunction || item2.IsTableFunction) && options2.LoadProcedure(item2))
                            {
                                string commandText = sqlBuilder.BuildObjectName(new StringBuilder(), new SqlObjectName(item2.ProcedureName, null, item2.CatalogName, item2.SchemaName, item2.PackageName)).ToString();
                                LoadProcedureTableSchema(dataConnection, options2, item2, commandText, list);
                            }

                            options2.ProcedureLoadingProgress(list2.Count, num++);
                        }
                    }
                    finally
                    {
                        if (!flag)
                        {
                            dataConnection.RollbackTransaction();
                        }
                    }
                }
                else
                {
                    list2 = new List<ProcedureSchema>();
                }

                List<ProcedureSchema> providerSpecificProcedures = GetProviderSpecificProcedures(dataConnection);
                if (providerSpecificProcedures != null)
                {
                    list2.AddRange(providerSpecificProcedures);
                }
            }
            else
            {
                list2 = new List<ProcedureSchema>();
            }

            return ProcessSchema(new DatabaseSchema
            {
                DataSource = GetDataSourceName(dataConnection),
                Database = GetDatabaseName(dataConnection),
                ServerVersion = connection.ServerVersion,
                Tables = list,
                Procedures = list2,
                ProviderSpecificTypeNamespace = GetProviderSpecificTypeNamespace(),
                DataTypesSchema = DataTypesSchema
            }, options2);
        }
}
MaceWindu commented 2 years ago

@KevinBurton could you try v4.1.1?

rkburton commented 2 years ago

Would you mind sending me instructions on how to downgrade?

sdanyliv commented 2 years ago

Why downgrade? Install latest version.

viceroypenguin commented 2 years ago

@rkburton - What are you trying to downgrade (l2db? linqpad?)?

We would like you to test LinqPAD6 or 7 with the latest version of the l2db driver. You can do this by adding the l2db driver again.

  1. Click "Add connection" in the connections list
  2. Click "View more drivers..." button
  3. Click the red "Update" link above the LINQ to DB driver.

Once you have updated the driver, you can close the various windows and go back to your query. At this point, please try to run your query again and let us know if you are still getting an NRE.

rkburton commented 2 years ago

I didn't check the version I have 4.1.0 and assumed that was the latest and it had a bug. If I delete the connection and try to add a driver now I get

image

This was after I selected 'View more drivers' to see the LINQ to DB driver from

image
sdanyliv commented 2 years ago

Why do not install latest version and check how it's going?

viceroypenguin commented 2 years ago

Version 4.1.1 has been released with a bugfix, which should address the NRE - this is why we want you to upgrade.

If you are unable to connect with api.nuget.org, then that is possibly a problem with your network that we would be unable to help with. I just tested, and was able to confirm that api.nuget.org is responding normally and the nuget service is working properly.

rkburton commented 2 years ago

Is this an option?

image
viceroypenguin commented 2 years ago

Yes, from this link, you can download the .lpx6 file and install it manually if you want.

viceroypenguin commented 2 years ago

Although the .lpx6 file relies on nuget as well, so you may not be able to install it if you are unable to access the nuget servers.

rkburton commented 2 years ago

I guess I am out of options the .lpx6 file wants to install 4.1.0 which it cannot find.

image
viceroypenguin commented 2 years ago

Can you access the nuget website (https://www.nuget.org/)? If not, then please check your network settings. From the images, it looks like there's an SSL issue between your machine and the nuget servers.

rkburton commented 2 years ago

Clicking on that linke I get

image
viceroypenguin commented 2 years ago

I'm afraid I don't know how to help you with your SSL issues. You might check that you are using the latest version of LinqPAD and that your version of Windows is up to date. Also, check if you have a proxy that it is not breaking the connection.

viceroypenguin commented 2 years ago

@rkburton - were you able to get your SSL issues resolved and test using v4.1.1?

MaceWindu commented 1 year ago

driver was completely rewritten. Open new issue if it still persists