docintelapp / DocIntel

Open Source Platform for storing, organizing, and searching documents related to cyber threats
https://docintel.org
Other
154 stars 24 forks source link

Fails to analyze some documents #78

Open ancailliau opened 1 year ago

ancailliau commented 1 year ago
2023-06-28 08:53:16.6715 [ERROR] [DocIntel.Core.Utils.DocumentAnalyzerUtility]    at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
   at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
   at System.Net.Sockets.Socket.Connect(IPAddress[] addresses, Int32 port)
--- End of stack trace from previous location ---
   at System.Net.Sockets.Socket.Connect(IPAddress[] addresses, Int32 port)
   at System.Net.Sockets.Socket.Connect(String host, Int32 port)
   at System.Net.Sockets.TcpClient.Connect(String hostname, Int32 port)
   at System.Net.Sockets.TcpClient..ctor(String hostname, Int32 port)
   at Synsharp.Telepath.Link.ConnectAsync() in /src/synsharp/Synsharp.Telepath/Link.cs:line 112
   at Synsharp.Telepath.Link.Connect(String host, Int32 port, LinkInfo linkInfo, ILogger logger) in /src/synsharp/Synsharp.Telepath/Link.cs:line 55
   at Synsharp.Telepath.Proxy.InitPoolLink() in /src/synsharp/Synsharp.Telepath/Proxy.cs:line 118
   at Synsharp.Telepath.Proxy.GetPoolLink() in /src/synsharp/Synsharp.Telepath/Proxy.cs:line 111
   at Synsharp.Telepath.Proxy.TaskV2(Todo todo, String name) in /src/synsharp/Synsharp.Telepath/Proxy.cs:line 413
   at CallSite.Target(Closure, CallSite, Object)
   at Synsharp.Telepath.Proxy.CallStormAsync(String query, StormOps opts) in /src/synsharp/Synsharp.Telepath/Proxy.cs:line 509
   at Synsharp.Telepath.Proxy.CallStormAsync[T](String query, StormOps opts) in /src/synsharp/Synsharp.Telepath/Proxy.cs:line 516
   at Synsharp.Telepath.Helpers.ViewHelper.List() in /src/synsharp/Synsharp.Telepath/Helpers/ViewHelper.cs:line 89
   at DocIntel.Core.Utils.Observables.SynapseRepository.GetViewAsync(Document document, Boolean unmerged) in /src/DocIntel.Core/Utils/Observables/SynapseRepository.cs:line 268
   at DocIntel.Core.Utils.Observables.SynapseRepository.Merge(Document document, Boolean delete) in /src/DocIntel.Core/Utils/Observables/SynapseRepository.cs:line 181
   at DocIntel.Core.Utils.DocumentAnalyzerUtility.Analyze(Guid documentId, AmbientContext ambientContext) in /src/DocIntel.Core/Utils/DocumentAnalyzerUtility.cs:line 229
ancailliau commented 1 year ago
2023-06-28 09:41:37.7434 [ERROR] [DocIntel.Core.Utils.DocumentAnalyzerUtility]    at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Synsharp.Telepath.Common.Result(Object reply) in /src/synsharp/Synsharp.Telepath/TelepathTask.cs:line 31
   at Synsharp.Telepath.Proxy.TaskV2(Todo todo, String name) in /src/synsharp/Synsharp.Telepath/Proxy.cs:line 413
   at CallSite.Target(Closure, CallSite, Object)
   at Synsharp.Telepath.Proxy.CallStormAsync(String query, StormOps opts) in /src/synsharp/Synsharp.Telepath/Proxy.cs:line 509
   at Synsharp.Telepath.Helpers.ViewHelper.Delete(String iden) in /src/synsharp/Synsharp.Telepath/Helpers/ViewHelper.cs:line 109
   at DocIntel.Core.Utils.Observables.SynapseRepository.Merge(Document document, Boolean delete) in /src/DocIntel.Core/Utils/Observables/SynapseRepository.cs:line 181
   at DocIntel.Core.Utils.DocumentAnalyzerUtility.Analyze(Guid documentId, AmbientContext ambientContext) in /src/DocIntel.Core/Utils/DocumentAnalyzerUtility.cs:line 229
ancailliau commented 1 year ago
2023-06-28 10:06:37.3918 [ERROR] [Microsoft.EntityFrameworkCore.Database.Command] Failed executing DbCommand (0ms) [Parameters=[@p19='?' (DbType = Guid), @p0='?' (DbType = Guid), @p1='?' (DbType = DateTime), @p2='?', @p3='?' (DbType = DateTime), @p4='?', @p5='?' (DbType = Object), @p6='?' (DbType = DateTime), @p7='?', @p8='?', @p9='?', @p10='?' (DbType = DateTime), @p11='?' (DbType = Int32), @p12='?', @p13='?' (DbType = Guid), @p14='?', @p15='?' (DbType = Int32), @p16='?' (DbType = Guid), @p17='?', @p18='?', @p39='?' (DbType = Guid), @p20='?' (DbType = Guid), @p21='?' (DbType = DateTime), @p22='?', @p23='?' (DbType = DateTime), @p24='?', @p25='?' (DbType = Object), @p26='?' (DbType = DateTime), @p27='?', @p28='?', @p29='?', @p30='?' (DbType = DateTime), @p31='?' (DbType = Int32), @p32='?', @p33='?' (DbType = Guid), @p34='?', @p35='?' (DbType = Int32), @p36='?' (DbType = Guid), @p37='?', @p38='?'], CommandType='Text', CommandTimeout='30']
2023-06-28 10:06:37.3943 [ERROR] [Microsoft.EntityFrameworkCore.Update] An exception occurred in the database while saving changes for context type 'DocIntel.Core.Models.DocIntelContext'.
Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while saving the entity changes. See the inner exception for details.
DETAIL: Detail redacted as it may contain sensitive data. Specify 'Include Error Detail' in the connection string to include this information.
    Severity: ERROR
    Detail: Detail redacted as it may contain sensitive data. Specify 'Include Error Detail' in the connection string to include this information.
ancailliau commented 1 year ago

2023-06-29 11:49:27.6566 [ERROR] [DocIntel.Core.Utils.DocumentAnalyzerUtility] Could not extract observable in file 'ad22dd46-644a-4743-93da-c052d3b43532' on document '988a5a79-ee20-443e-b2ca-06089664d204': The given key 'mesg' was not present in the dictionary.

ancailliau commented 1 year ago
2023-06-29 12:10:42.2987 [ERROR] [DocIntel.Core.Utils.DocumentAnalyzerUtility] Could not extract observable in file '7c203b18-1bd5-495a-9338-46b8d56b9e54' on document '3c5b0bbb-0579-4484-a7d9-7154d753d22e': 'System.Array' does not contain a definition for 'Data'
ancailliau commented 1 year ago

Merge a non-existing view makes the process not renew its telepath proxy

ancailliau commented 1 year ago

2023-06-30 08:04:30.9899 [ERROR] [DocIntel.Core.Utils.DocumentAnalyzerUtility] Could not extract observable in file '3e508719-0fe4-4664-bd5a-2a05b6833fc2' on document '9a866913-9a84-413a-8b1e-9933fedbf128': 'System.Array' does not contain a definition for 'Data'

ancailliau commented 1 year ago

2023-06-30 11:58:17.7552 [ERROR] [DocIntel.Core.Utils.DocumentAnalyzerUtility] Could not extract observable in file '321caf03-2d48-43c7-95ac-39d79ead9e3b' on document 'b3c25b44-d33f-40f3-89f3-bc58dc9944be': An item with the same key has already been added. Key: _di.workflow.review