Open klemen-df opened 3 weeks ago
Hey @klemen-df! I'm here to help you with any bugs, questions, or contributions you have for the project. Let me know how I can assist you!
To resolve the error "Data too long for column 'component' at row 209" encountered during the SonarQube scraping process in version v1.0.0-beta10, you need to modify the length of the 'component' column in the relevant tables. The provided migration script 20231007_modify_component_length.go
changes the column type to varchar(500)
for the component
field in the _tool_sonarqube_hotspots
, _tool_sonarqube_issue_code_blocks
, and _tool_sonarqube_issues
tables.
Here is the relevant migration script:
package migrationscripts
import (
"github.com/apache/incubator-devlake/core/context"
"github.com/apache/incubator-devlake/core/dal"
"github.com/apache/incubator-devlake/core/errors"
"github.com/apache/incubator-devlake/core/plugin"
"github.com/apache/incubator-devlake/helpers/migrationhelper"
)
var _ plugin.MigrationScript = (*modifyComponentLength)(nil)
type modifyComponentLength struct{}
type sonarqubeHotspot20231007 struct {
Component string `gorm:"index;type:varchar(500)"`
}
func (sonarqubeHotspot20231007) TableName() string {
return "_tool_sonarqube_hotspots"
}
type sonarqubeIssueCodeBlock20231007 struct {
Component string `gorm:"index;type:varchar(500)"`
}
func (sonarqubeIssueCodeBlock20231007) TableName() string {
return "_tool_sonarqube_issue_code_blocks"
}
type sonarqubeIssue20231007 struct {
Component string `gorm:"index;type:varchar(500)"`
}
func (sonarqubeIssue20231007) TableName() string {
return "_tool_sonarqube_issues"
}
func (script *modifyComponentLength) Up(basicRes context.BasicRes) errors.Error {
db := basicRes.GetDal()
err := migrationhelper.ChangeColumnsType[sonarqubeHotspot20231007](
basicRes,
script,
sonarqubeHotspot20231007{}.TableName(),
[]string{"component"},
func(tmpColumnParams []interface{}) errors.Error {
return db.UpdateColumn(
&sonarqubeHotspot20231007{},
"component",
dal.DalClause{Expr: " ? ", Params: tmpColumnParams},
dal.Where("? != '' ", tmpColumnParams...),
)
},
)
if err != nil {
return err
}
err = migrationhelper.ChangeColumnsType[sonarqubeIssueCodeBlock20231007](
basicRes,
script,
sonarqubeIssueCodeBlock20231007{}.TableName(),
[]string{"component"},
func(tmpColumnParams []interface{}) errors.Error {
return db.UpdateColumn(
&sonarqubeIssueCodeBlock20231007{},
"component",
dal.DalClause{Expr: " ? ", Params: tmpColumnParams},
dal.Where("? != '' ", tmpColumnParams...),
)
},
)
if err != nil {
return err
}
err = migrationhelper.ChangeColumnsType[sonarqubeIssue20231007](
basicRes,
script,
sonarqubeIssue20231007{}.TableName(),
[]string{"component"},
func(tmpColumnParams []interface{}) errors.Error {
return db.UpdateColumn(
&sonarqubeIssue20231007{},
"component",
dal.DalClause{Expr: " ? ", Params: tmpColumnParams},
dal.Where("? != '' ", tmpColumnParams...),
)
},
)
if err != nil {
return err
}
return nil
}
func (*modifyComponentLength) Version() uint64 {
return 20231007145127
}
func (*modifyComponentLength) Name() string {
return "modify component type to varchar(500)"
}
This script increases the length of the component
column to 500 characters, which should resolve the "Data too long for column 'component'" error. Additionally, if you encounter similar issues with other columns or tables, you can refer to the Configuration and Blueprint Troubleshooting documentation for more detailed solutions and workarounds [1][2].
To continue talking to Dosu, mention @dosu.
Search before asking
What happened
when running scraping I get this error:
subtask convertIssueCodeBlocks ended unexpectedly Wraps: (2) error adding result to batch (500) Wraps: (3) Error 1406 (22001): Data too long for column 'component' at row 209 (500) Wraps: (4) Error 1406 (22001): Data too long for column 'component' at row 209 Error types: (1) *hintdetail.withDetail (2) *hintdetail.withDetail (3) *hintdetail.withDetail (4) *mysql.MySQLError
What do you expect to happen
The process should end without an error.
How to reproduce
Add SonarQube project and run.
Anything else
No response
Version
v1.0.0-beta10
Are you willing to submit PR?
Code of Conduct