encoding: add AppendText and AppendBinary

dsnet commented 1 year ago

The MarshalText and MarshalBinary methods are a trash factory. They create a short-lived string that is almost always garbage collectable shortly after creation. Furthermore, the caller almost always copied the result into another buffer, resulting in further waste.

I propose the addition of the following interfaces to the "encoding" package:

// TextAppender is the interface implemented by an object
// that can append the textual representation of itself.
// If a type implements both [TextAppender] and [TextMarshaler],
// then v.MarshalText() must be semantically identical to v.AppendText(nil).
type TextAppender interface {
    // AppendText appends the textual representation of itself to the end of b
    // (allocating a larger slice if necessary) and returns the updated slice.
    //
    // Implementations must not retain b, nor mutate any bytes within b[:len(b)].
    AppendText(b []byte) ([]byte, error)
}

type BinaryAppender interface {
    AppendBinary([]byte) ([]byte, error)
}

In addition to the above, we would update all types in the stdlib that implement MarshalText or MarshalBinary to have the Append equivalent (and switch the Marshal implementation to just call the Append variant). Also, we would teach all stdlib libraries that looks for TextMarshaler or BinaryMarshaler to also look for the append (and unmarshal) variant.

\cc @bradfitz @maisem

dsnet commented 1 year ago

This is motivated by netip.Addr.MarshalText being a non-trivial amount of slowdown in a large JSON object.

dsnet commented 1 year ago

Note that I did not propose json.Appender. There are other possible API designs that allow for streaming JSON encoding and decoding which would obviate the need for it. A separate proposal or discussion will be made for this in the future.

dsnet commented 1 year ago

Analyzing all code on the module proxy from 2023-07-01:

186 methods named "AppendText"
- 0 methods match the proposed signatures
25 methods named "AppendBinary"
- 3 methods match the proposed signatures
  - transaction.ExtraTag.AppendBinary
  - transaction.ExtraTags.AppendBinary
  - transaction.Outputs.AppendBinary
  - all 3 also have a MarshalBinary method, where AppendBinary(nil) is identical to MarshalBinary

I do not believe there are any backwards compatibility issues with this change.

barista.run@v0.0.0-20230630192130-1694cee7a398/pango/pango.go:95: func (n *Node) AppendText(texts ...string) *Node {
bitbucket.org/afiszone/gotk3@v0.0.0-20200317205801-68a9308a074c/gtk/combo_box.go:263: func (v *ComboBoxText) AppendText(text string) {
code.afis.me/gotk3@v0.0.0-20201225185300-6238108e03d9/gtk/combo_box.go:354: func (v *ComboBoxText) AppendText(text string) {
code.aliyun.com/newrank-tools/document@v0.0.3/api/markdown/markdown.go:47: func (md *Markdown) AppendText(content string, params ...bool) *Markdown {
codeberg.org/japh/psv@v0.2.0/table.go:149: func (tbl *Table) AppendText(textLines ...string) {
git.gammaspectra.live/P2Pool/p2pool-observer@v0.0.0-20230605214634-bad1e302c8c8/monero/transaction/extra.go:54: func (t *ExtraTags) AppendBinary(preAllocatedBuf []byte) (buf []byte, err error) {
git.gammaspectra.live/P2Pool/p2pool-observer@v0.0.0-20230605214634-bad1e302c8c8/monero/transaction/extra.go:124: func (t *ExtraTag) AppendBinary(preAllocatedBuf []byte) ([]byte, error) {
git.gammaspectra.live/P2Pool/p2pool-observer@v0.0.0-20230605214634-bad1e302c8c8/monero/transaction/output.go:78: func (s *Outputs) AppendBinary(preAllocatedBuf []byte) (data []byte, err error) {
git.gammaspectra.live/P2Pool/p2pool-observer@v0.0.0-20230605214634-bad1e302c8c8/p2pool/sidechain/sidedata.go:52: func (b *SideData) AppendBinary(preAllocatedBuf []byte, version ShareVersion) (buf []byte, err error) {
git.milar.in/milarin/tui@v0.0.21/rune.go:46: func (t *Text) AppendText(txt *Text) {
git.tordarus.net/Tordarus/tui@v0.0.15/rune.go:46: func (t *Text) AppendText(txt *Text) {
git.wit.org/wit/gui@v0.8.6/common.go:91: func (n *Node) AppendText(str string) {
gitee.com/scpro/gotk3@v1.0.1/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
gitee.com/zhouhailin/hutool-go@v0.0.2/hutool/io/io_util.go:27: func (f *IoUtil) AppendText(pathname string, text string) error {
github.com/45638654564654/appoptics-apm-go@v0.0.0-20221005200510-e098d2000040/v1/ao/internal/bson/bson.go:43: func (b *Buffer) AppendBinary(k string, v []byte) {
github.com/474420502/libxml2@v0.0.0-20200807040518-4ef6186ae68c/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/BlackMocca/mongo-go-driver@v1.0.4/x/bsonx/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
github.com/BlackMocca/mongo-go-driver@v1.0.4/x/bsonx/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
github.com/Existed/libxml2@v0.0.0-20190422014600-3dcec2c44e24/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/FabianWe/etherpadlite-golang@v1.1.0/etherpadlite.go:308: func (pad *EtherpadLite) AppendText(ctx context.Context, padID, text interface{}) (*Response, error) {
github.com/Gipcomp/winapi@v0.0.0-20211028125119-f5f50e8984c7/textedit.go:245: func (te *TextEdit) AppendText(value string) {
github.com/GoTyro/walk@v0.0.0-20220112065045-02bb5eb37497/textedit.go:241: func (te *TextEdit) AppendText(value string) {
github.com/GoTyro/walk@v0.0.0-20220112065045-02bb5eb37497/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/KemalovMaulen/libxml2@v1.0.0/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/Skarm/walk@v0.0.0-20150220080041-98cb48b69104/textedit.go:94: func (te *TextEdit) AppendText(value string) {
github.com/Skarm/walk@v0.0.0-20150220080041-98cb48b69104/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/StephanVerbeeck/walk@v0.0.0-20230202143337-d63a2e470fda/textedit.go:217: func (te *TextEdit) AppendText(value string) {
github.com/StephanVerbeeck/walk@v0.0.0-20230202143337-d63a2e470fda/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/adejoux/nmon2influxdb@v2.1.7-0.20190605081557-84e3405360f7+incompatible/nmon/nmon.go:42: func (nmon *Nmon) AppendText(text string) {
github.com/admpub/spider@v0.0.0-20171028061754-3a892f2f9e33/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/agl/go-gtk@v0.0.0-20151214221645-91c1edb38c24/gtk/gtk.go:5447: func (v *GtkComboBoxText) AppendText(text string) {
github.com/anoshenko/rui@v0.13.0/editView.go:357: func (edit *editViewData) AppendText(text string) {
github.com/appoptics/appoptics-apm-go@v1.16.0/v1/ao/internal/bson/bson.go:43: func (b *Buffer) AppendBinary(k string, v []byte) {
github.com/auroralaboratories/gotk3@v0.0.0-20170323014032-52f9ba079f44/gtk/combo_box.go:229: func (v *ComboBoxText) AppendText(text string) {
github.com/bakharal/mongo-go-driver@v1.10.11/x/bsonx/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
github.com/bakharal/mongo-go-driver@v1.10.11/x/bsonx/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
github.com/bennicholls/burl-E@v0.0.0-20190708032244-24ccafc7d477/burl/textbox.go:29: func (t *Textbox) AppendText(txt string) {
github.com/blorticus/tpcli@v0.3.1/tpcli.go:391: func (panel *outputPanel) AppendText(s string) {
github.com/bontibon/go-pipeglade@v0.0.0-20150212114355-0ac764b2f9e2/manipulation.go:141: func (ui *UI) AppendText(widget, text string) {
github.com/bytbox/gogobject@v0.0.0-20120219142239-4e70a8595dcf/gtk-3.0/gtk.go:14947: func (this0 *ComboBoxText) AppendText(text0 string) {
github.com/chai2010/walk@v0.0.2/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/chai2010/walk@v0.0.2/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/clockworkpi/LauncherGoDev@v0.0.0-20230126075719-2743d6b9dc1e/sysgo/UI/textarea.go:80: func (self *Textarea) AppendText(alphabet string) {
github.com/clockworksoul/mediawiki@v0.0.5/edit.go:259: func (w *EditClient) AppendText(s string) *EditClient {
github.com/cloudfly/alarmd@v0.0.0-20191209053936-d369a3264630/pkg/events/channel.go:61: func (spec ChannelSpec) AppendText(data map[string]interface{}) string {
github.com/cloudfly/ecenter@v0.0.0-20191220053526-dfd69a52662f/pkg/events/channel.go:61: func (spec ChannelSpec) AppendText(data map[string]interface{}) string {
github.com/cloudradar-monitoring/frontman@v0.0.0-20210924115943-7c61842a19b8/pkg/winui/winui_logview.go:64: func (lv *LogView) AppendText(value string) {
github.com/coyim/gotk3adapter@v0.0.2/gtk_mock/combo_box_text.go:7: func (*MockComboBoxText) AppendText(v1 string) {
github.com/coyim/gotk3adapter@v0.0.2/gtka/combo_box_text.go:31: func (v *comboBoxText) AppendText(v1 string) {
github.com/coyim/gotk3mocks@v0.0.2/gtk/combo_box_text.go:7: func (m *MockComboBoxText) AppendText(v1 string) {
github.com/d2r2/go-gtk@v0.0.0-20131218031741-e35a6dfb5a85/gtk/gtk.go:5223: func (v *ComboBox) AppendText(text string) {
github.com/d2r2/go-gtk@v0.0.0-20131218031741-e35a6dfb5a85/gtk/gtk.go:5309: func (v *ComboBoxText) AppendText(text string) {
github.com/d2r2/gotk3@v0.0.0-20211023153135-0ceb344f1671/gtk/combobox.go:240: func (v *ComboBoxText) AppendText(text string) {
github.com/davidli2010/gobson_exp@v0.0.0-20161004043538-c651a288efad/bson/bsonarraybuilder.go:103: func (a *BsonArrayBuilder) AppendBinary(value Binary) *BsonArrayBuilder {
github.com/davidli2010/gobson_exp@v0.0.0-20161004043538-c651a288efad/bson/bsonbuilder.go:183: func (b *BsonBuilder) AppendBinary(name string, value Binary) *BsonBuilder {
github.com/diamondburned/gotk4/pkg@v0.0.5/gtk/v3/gtkcomboboxtext.go:235: func (comboBox *ComboBoxText) AppendText(text string) {
github.com/diamondburned/gotk4/pkg@v0.0.5/gtk/v4/gtkcomboboxtext.go:213: func (comboBox *ComboBoxText) AppendText(text string) {
github.com/dimansk12/libxml2@v1.0.0/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_darwin.go:84776: func (arg1 SwigClassTextEntry) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_darwin.go:245455: func (arg1 SwigClassStyledTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_darwin.go:290453: func (arg1 SwigClassRichTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_linux.go:84736: func (arg1 SwigClassTextEntry) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_linux.go:245543: func (arg1 SwigClassStyledTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_linux.go:288912: func (arg1 SwigClassRichTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_windows.go:85111: func (arg1 SwigClassTextEntry) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_windows.go:246501: func (arg1 SwigClassStyledTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_windows.go:289885: func (arg1 SwigClassRichTextCtrl) AppendText(arg2 string) {
github.com/ebitenui/ebitenui@v0.5.4/widget/textarea.go:336: func (l *TextArea) AppendText(value string) {
github.com/electricface/go-gir@v0.0.0-20220119144140-f2fd0857a7da/gtk-3.0/gtk_auto.go:15477: func (v ComboBoxText) AppendText(text string) {
github.com/friendstechday/codeDocToMarkdown@v0.0.0-20150523005008-fc62c7cac2de/converter/java/javaDoc/comment.go:11: func (c *Comment) AppendText(text string) {
github.com/friendstechday/codeDocToMarkdown@v0.0.0-20150523005008-fc62c7cac2de/converter/java/javaDoc/param.go:13: func (p *Param) AppendText(text string) {     
github.com/gangming/walk@v0.0.0-20220825033616-0c0e321abef9/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/gangming/walk@v0.0.0-20220825033616-0c0e321abef9/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/gimalay/binx@v0.0.0-20200623152039-84df950e8e35/binx.go:78: func (e *page) AppendBinary(data []byte) (bool, error) {
github.com/gimalay/binx@v0.0.0-20200623152039-84df950e8e35/binx.go:94: func (c *Count) AppendBinary([]byte) error {
github.com/gimalay/binx@v0.0.0-20200623152039-84df950e8e35/binx_test.go:37: func (e *indexable) AppendBinary(data []byte) (bool, error)  { return false, json.Unmarshal(data, e) }
github.com/gimalay/binx@v0.0.0-20200623152039-84df950e8e35/binx_test.go:52: func (e *indexableSlice) AppendBinary(data []byte) (bool, error) {
github.com/godot-go/godot-go@v0.0.0-20230312082217-9d5a86402ae8/pkg/gdextension/classes.gen.go:341178: func (cx *RichTextLabelImpl) AppendText(
github.com/gofaith/walk@v0.0.0-20200803081117-681a3effcc77/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/gofaith/walk@v0.0.0-20200803081117-681a3effcc77/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/goki/gi@v1.3.19/giv/textbuf.go:707: func (tb *TextBuf) AppendText(text []byte, signal bool) *textbuf.Edit {
github.com/gontikr99/bidbot2@v0.0.0-20190924173835-de946e3c6c1e/controller/gui/logview.go:60: func (lv *LogView) AppendText(value string) {
github.com/gotk3/gotk3@v0.6.2/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
github.com/hegeng1212/pholcus@v0.0.0-20230627120450-78db5a490cf2/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/henrylee2cn/pholcus@v1.3.4/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/ianatha/walk@v0.0.0-20200225051714-2c8e61d8d5e8/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/ianatha/walk@v0.0.0-20200225051714-2c8e61d8d5e8/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/icowan/walk@v0.0.0-20201223064526-8326e90d8f36/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/icowan/walk@v0.0.0-20201223064526-8326e90d8f36/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/ingmardrewing/gomic@v0.0.0-20170621162852-b6f2c68bbe53/fs/html.go:116: func (n *htmlNode) AppendText(txt string) node {
github.com/ionous/sashimi@v0.0.0-20160915000703-9034458fa291/metal/lists.go:36: func (ar arrayValues) AppendText(v string) error {
github.com/ionous/sashimi@v0.0.0-20160915000703-9034458fa291/metal/panicValue.go:31: func (p panicValue) AppendText(string) error {
github.com/ionous/sashimi@v0.0.0-20160915000703-9034458fa291/runtime/internal/gameList.go:126: func (l *gameList) AppendText(v string) {
github.com/ionous/sashimi@v0.0.0-20160915000703-9034458fa291/runtime/internal/nullList.go:13: func (n nullList) AppendText(string)         {}
github.com/iquanxin/walk@v0.0.0-20230418091138-4b9082def936/textedit.go:241: func (te *TextEdit) AppendText(value string) {
github.com/jannson/walk@v0.0.0-20221010063135-328f21216f11/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/jannson/walk@v0.0.0-20221010063135-328f21216f11/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/jason-wj/pholcus@v0.0.0-20230628053201-2b89c876f2ed/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/joeirimpan/libxml2@v1.0.4/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/johnbartholomew/go-gtk@v0.0.0-20130206210405-e86225ead39e/gtk/gtk.go:4955: func (v *ComboBox) AppendText(text string) {
github.com/johnbartholomew/go-gtk@v0.0.0-20130206210405-e86225ead39e/gtk/gtk.go:5041: func (v *ComboBoxText) AppendText(text string) {
github.com/jopbrown/gtk-sugar@v0.0.0-20191124065551-9a86599f5831/lib/gtk/combo.go:42: func (obj *ComboBoxText) AppendText(text string) {
github.com/jopbrown/gtk-sugar@v0.0.0-20191124065551-9a86599f5831/lib/gtk/combo.gtk2.go:10: func (obj *ComboBox) AppendText(text string) {
github.com/justinyaoqi/pholcus@v0.0.0-20221204035437-fbc1b0707612/gui/logview.go:65: func (lv *LogView) AppendText(value string) {
github.com/jzs/walk@v0.0.0-20181112095039-c7d36e6fcacf/textedit.go:147: func (te *TextEdit) AppendText(value string) {
github.com/jzs/walk@v0.0.0-20181112095039-c7d36e6fcacf/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/karashiiro/kartlobby@v0.0.0-20221019160947-d0596f565030/pkg/colortext/builder.go:11: func (b *ColorTextBuilder) AppendText(text string) *ColorTextBuilder {
github.com/kckrinke/go-gtk@v0.0.0-20180506212049-4cde83b81e6e/gtk/gtk.go:6126: func (v *ComboBox) AppendText(text string) {
github.com/kckrinke/go-gtk@v0.0.0-20180506212049-4cde83b81e6e/gtk/gtk.go:6221: func (v *ComboBoxText) AppendText(text string) {
github.com/ktye/ui@v0.0.0-20210211114936-94158a1502e8/editor/edit.go:47: func (e *Edit) AppendText(s string) {
github.com/kumakichi/walk@v0.0.0-20150122092410-7e83dfcca18a/textedit.go:94: func (te *TextEdit) AppendText(value string) {
github.com/kumakichi/walk@v0.0.0-20150122092410-7e83dfcca18a/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/kvark128/walk@v0.0.0-20230207150301-925603f55833/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/kvark128/walk@v0.0.0-20230207150301-925603f55833/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/lazyshot/gotk3@v0.0.0-20150809002019-08a6d4f8db4c/gtk/gtk.go:2629: func (v *ComboBoxText) AppendText(text string) {
github.com/leaanthony/go-gtk@v0.0.0-20211127021733-72e2f9c2edc1/gtk/gtk.go:6189: func (v *ComboBox) AppendText(text string) {
github.com/leaanthony/go-gtk@v0.0.0-20211127021733-72e2f9c2edc1/gtk/gtk.go:6284: func (v *ComboBoxText) AppendText(text string) {
github.com/lestrrat/go-libxml2@v0.0.0-20180221004755-bb78334e2019/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/lestrrat-go/libxml2@v0.0.0-20210328064523-a6d61dc4e799/dom/node_element.go:64: func (n *Element) AppendText(s string) error {
github.com/liwh011/gonebot@v0.2.1/message.go:44: func (m *Message) AppendText(t string) {
github.com/loyalpartner/go-nigori@v0.0.5/stream.go:31: func (n *nigoriStream) AppendText(str string) {
github.com/lufuhu/gotk3@v0.6.3/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
github.com/lxn/walk@v0.0.0-20210112085537-c389da54e794/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/lxn/walk@v0.0.0-20210112085537-c389da54e794/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/mad-day/bsonbox@v0.0.0-20211002061631-2f125da2d253/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
github.com/mad-day/bsonbox@v0.0.0-20211002061631-2f125da2d253/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
github.com/marcusolsson/passtray@v0.0.0-20150813080859-7d312850bdc9/Godeps/_workspace/src/github.com/mattn/go-gtk/gtk/gtk.go:5310: func (v *ComboBox) AppendText(text string) {
github.com/marcusolsson/passtray@v0.0.0-20150813080859-7d312850bdc9/Godeps/_workspace/src/github.com/mattn/go-gtk/gtk/gtk.go:5396: func (v *ComboBoxText) AppendText(text string) {
github.com/mardukbp/atk@v1.0.3/tk/text.go:410: func (w *Text) AppendText(text string) error {
github.com/mattn/go-gtk@v0.0.0-20230419095350-e099c1bf7abc/gtk/gtk.go:6188: func (v *ComboBox) AppendText(text string) {
github.com/mattn/go-gtk@v0.0.0-20230419095350-e099c1bf7abc/gtk/gtk.go:6283: func (v *ComboBoxText) AppendText(text string) {
github.com/maurofran/hamcrest4go@v0.2.2/matcher/description.go:70: func (d Description) AppendText(text string) {
github.com/mjm/mpsanity@v0.0.0-20211020031355-e32cecb0bc89/block/builder.go:96: func (b *Builder) AppendText(text string) {
github.com/mrbear258/gotk3@v1.0.0/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
github.com/muumlover/walk@v0.0.0-20181212150522-fb9fbbcc6391/textedit.go:151: func (te *TextEdit) AppendText(value string) {
github.com/muumlover/walk@v0.0.0-20181212150522-fb9fbbcc6391/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/natemaia/gotk3@v0.6.4/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
github.com/norisatir/go-gtk3@v0.0.0-20120410091437-f6c2ba810f40/gtk3/gtk3.go:11356: func (self *ComboBoxText) AppendText(text string) {
github.com/novln/soba@v0.0.0-20201126173112-c6f1bc371303/encoder_test.go:140: func (encoder *TestEncoder) AppendBinary(value []byte) {}
github.com/novln/soba@v0.0.0-20201126173112-c6f1bc371303/encoder/json/types.go:434: func (encoder *Encoder) AppendBinary(value []byte) {
github.com/oldkingnearby/walk@v1.0.0/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/oldkingnearby/walk@v1.0.0/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/pauldub/go-gtk3@v0.0.0-20120812130113-4604e9480944/gtk3/gtk3.go:11358: func (self *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-2.24.go:16816: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.0.go:18683: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.10.go:19473: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.12.go:19619: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.14.go:19877: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.16.go:19976: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.18.go:20013: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.2.go:18901: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.20.go:20182: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.22.26.go:20199: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.22.29.go:20217: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.22.6.go:20190: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.22.go:20190: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.4.go:19167: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.6.go:19278: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.8.go:19293: func (recv *ComboBoxText) AppendText(text string) {
github.com/pietroglyph/walk@v0.0.0-20180728074727-d9c93ed8784b/textedit.go:147: func (te *TextEdit) AppendText(value string) {
github.com/pietroglyph/walk@v0.0.0-20180728074727-d9c93ed8784b/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/pirogom/walk@v0.0.0-20230128105705-492944454fc3/textedit.go:241: func (te *TextEdit) AppendText(value string) {
github.com/planetscale/log@v0.0.0-20230320220358-9d4ed51abd8d/encoder.go:195: func (enc *prettyEncoder) AppendBinary(value []byte) {
github.com/pritunl/mongo-go-driver@v1.11.3/x/bsonx/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
github.com/pritunl/mongo-go-driver@v1.11.3/x/bsonx/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
github.com/pwiecz/atk@v0.0.0-20210227100443-cb0bf65e43e2/tk/text.go:410: func (w *Text) AppendText(text string) error {
github.com/pzartem/libxml2@v0.0.0-20200831082948-c3d9892b3a0f/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/qsirwyk/walk@v1.1.1/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/qsirwyk/walk@v1.1.1/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/raceresult/walk@v0.0.4/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/readykit/gd@v0.0.0-20221015225739-25a193a18dd7/api.go:25532: func (gdClass RichTextLabel) AppendText(bbcode string)  { methodCall[struct{}](gdClass.obj.get(), methodRichTextLabel[76], &bbcode) }
github.com/reusee/ggir@v0.0.0-20141013104036-88ce4855c267/gtk/gtk_traits.go:5592: func (self *TraitComboBoxText) AppendText(text string) {
github.com/richardwilkes/cef@v0.5.3/cef/Textfield_gen.go:111: func (d *Textfield) AppendText(text string) {
github.com/romychs/gotk3@v0.0.0-20210328092339-116e09e25f5e/gtk/combobox.go:240: func (v *ComboBoxText) AppendText(text string) {
github.com/rstshardware/walk@v0.0.1/textedit.go:241: func (te *TextEdit) AppendText(value string) {
github.com/rstshardware/walk@v0.0.1/examples/logview/logview.go:53: func (lv *LogView) AppendText(value string) {
github.com/scjtqs2/walk@v0.0.0-20211230061331-3545c43f16e8/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/scjtqs2/walk@v0.0.0-20211230061331-3545c43f16e8/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/tHinqa/outside-gtk2@v0.0.0-20131129203352-f05e200c0477/gtk/gtk_c.go:1059: func (c *ComboBox) AppendText(text string) { ComboBoxAppendText(c, text) }
github.com/tHinqa/outside-gtk2@v0.0.0-20131129203352-f05e200c0477/gtk/gtk_c.go:1109: func (c *ComboBoxText) AppendText(text string) { ComboBoxTextAppendText(c, text) }
github.com/tHinqa/outside-gtk2@v0.0.0-20131129203352-f05e200c0477/gtk/gtk_e.go:194: func (e *Entry) AppendText(text string)            { EntryAppendText(e, text) }
github.com/tailscale/walk@v0.0.0-20230502163740-ea8725bca6d8/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/tailscale/walk@v0.0.0-20230502163740-ea8725bca6d8/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/tiborvass/uniline@v0.0.0-20150528164013-b39ee5b15151/utils.go:58: func (t text) AppendText(n text) text {
github.com/timelessnesses/l4d2-fmsa-tk@v0.0.0-20230302131554-ce07a2f78b70/tk/text.go:410: func (w *Text) AppendText(text string) error {
github.com/tmaxmax/go-sse@v0.4.2/message.go:98: func (e *Message) AppendText(chunks ...string) {
github.com/turutcrane/cefingo@v0.5.9/capi/cefingo_gen.go:27863: func (self *CTextfieldT) AppendText(
github.com/vikulin/walk@v0.0.0-20210329133517-4f03df2e225a/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/vikulin/walk@v0.0.0-20210329133517-4f03df2e225a/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/visualfc/atk@v1.2.3/tk/text.go:410: func (w *Text) AppendText(text string) error {
github.com/wangch/walk@v0.0.0-20150802023425-cdcaffe9ed2d/textedit.go:94: func (te *TextEdit) AppendText(value string) {
github.com/wangch/walk@v0.0.0-20150802023425-cdcaffe9ed2d/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/wayf-dk/go-libxml2@v0.0.0-20210718205101-95691c57c1ac/dom/node_element.go:65: func (n *Element) AppendText(s string) error {
github.com/weili71/go-filex@v0.0.0-20230321132739-7a825b3f5691/filex.go:200: func (f *Filex) AppendText(file *os.File, text string) (int, error) {
github.com/whtiehack/wingui@v0.0.6/edit.go:50: func (e *Edit) AppendText(value string) {
github.com/winxxp/walk@v0.0.0-20200901104446-3e7ddcbafc7d/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/winxxp/walk@v0.0.0-20200901104446-3e7ddcbafc7d/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/xlplbo/walk@v0.0.0-20201127063646-ea1cc4883b4d/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/xlplbo/walk@v0.0.0-20201127063646-ea1cc4883b4d/examples/logview/logview.go:38: func (lv *LogView) AppendText(value string) {
github.com/yamnikov-oleg/go-gtk@v0.0.0-20160312170510-bc91c0fbef0c/gtk/gtk.go:5343: func (v *ComboBox) AppendText(text string) {
github.com/yamnikov-oleg/go-gtk@v0.0.0-20160312170510-bc91c0fbef0c/gtk/gtk.go:5430: func (v *ComboBoxText) AppendText(text string) {
github.com/zhangweiii/pholcus@v0.0.0-20200228101341-71bf9bad259b/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/zhangyiming748/sendEmailAlert@v0.0.14/alert.go:67: func (i *Info) AppendText(s string) {
github.com/zurek87/go-gtk3@v0.0.0-20171203223943-bec742635bb7/gtk3/gtk3.go:5867: func (v *ComboBoxText) AppendText(text string) {
github.com/zxjsdp/specimen-go@v1.9.4/gui/windows/logview.go:63: func (lv *LogView) AppendText(value string) {
gitlab.com/abibino-lab/mobal@v0.0.0-20170907170701-2d91a9572e70/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
gitlab.com/gitlab-org/security-products/analyzers/common/table@v0.0.0-20181219150649-131c639c2176/table.go:30: func (t *Table) AppendText(text string) {
gitlab.com/gitlab-org/security-products/analyzers/common/table/v2@v2.0.0-20201123204353-3743d5d8b4cc/table.go:30: func (t *Table) AppendText(text string) {
go.mongodb.org/mongo-driver@v1.12.0/x/bsonx/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
go.mongodb.org/mongo-driver@v1.12.0/x/bsonx/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
modernc.org/atk@v1.4.3/text.go:410: func (w *Text) AppendText(text string) error {

mateusz834 commented 1 year ago

This is motivated by netip.Addr.MarshalText being a non-trivial amount of slowdown in a large JSON object.

Just so you now there is netip.Addr.AppendTo, that does the same thing as MarshalText, but appends.

dsnet commented 1 year ago

There are unfortunately two problems with AppendTo:

It doesn't return an error, and so can't implement equivalent semantics as MarshalText
It's naming is ambiguous with regard to whether it does "binary" or "textual" marshaling

mateusz834 commented 1 year ago

@dsnet Yes I agree, just wanted to point that there will be now two methods that do the same thing on netip.Addr, probably after this proposal gets accepted the AppendTo should be deprecated.

mvdan commented 1 year ago

Scratching my head and wondering how this hasn't been proposed before :) sounds great to me.

flimzy commented 1 year ago

What are the possible security implications wrt:

// Implementations must not retain b, nor mutate any bytes within b[:len(b)].

I can imagine a malicious, or sloppy, library not following this rule, and introducing nastiness.

Given the other vectors for "nastiness" that already exist throughout the stdlib when using untrusted code, probably not a meaningful consideration, but it does seem a bit icky to me, to expose some other code's data to arbitrary code this way.

dsnet commented 1 year ago

Given the other vectors for "nastiness" that already exist throughout the stdlib when using untrusted code, probably not a meaningful consideration, but it does seem a bit icky to me, to expose some other code's data to arbitrary code this way.

Agreed. That's the strength of the MarshalText API. It's much harder to hold wrong.

That said, two of the top-three interfaces in Go by far are io.Reader and io.Writer, both of which expose a buffer to the implementation and document:

Implementations must not retain p.

I don't think we're making the situation any worse since encoding.AppendText will be multiple orders of magnitude less popular than io.Reader or io.Writer.

seebs commented 1 year ago

This seems very desireable, the large allocations can be a problem. Although I think in a lot of the cases that are particularly messy, append might not help, because i really need to do an encoder to an io.Writer? But I would still very much like to have this, because iterative loops that are marshaling a bunch of things but only need one at a time are pretty common.

mateusz834 commented 1 year ago

@seebs That is true that something like this would be a nice alternative.

type TextWriter interface {
    WriteText(b io.Writer) error
}

But, this as as side effect would cause another bunch of memory allocations inside each implementation of WriteText, because escape analysis is not great for this use case.

dsnet commented 1 year ago

Any encoder writing eventually to an io.Writer almost certainly have an intermediate buffer on hand that it can pass to AppendText and then pass eventually to io.Writer.Write.

yerden commented 1 year ago

Upvoting. I cannot imagine any heavily loaded application tinkering with bytes and text that wouldn't implement an interface of similar semantics.

mitar commented 1 year ago

nor mutate any bytes within b[:len(b)].

Why, performance wise, would that not be allowed? Maybe we can open additional use cases if we do allow that?

dsnet commented 1 year ago

Why, performance wise, would that not be allowed?

For a function named "Append", it just seems highly suspicious for it to mutate data within b[:len(b)]. The verb "append" implies only modifying data within b[len(b):cap(b)].

A performance argument could be made in the reverse. The caller of AppendText could in theory spin off a goroutine to do some form of read-only validation within b[:len(b)]. Mutating that section would result in a race condition.

mitar commented 1 year ago

For a function named "Append", it just seems highly suspicious for it to mutate data within b[:len(b)]. The verb "append" implies only modifying data within b[len(b):cap(b)].

So should we rename it then? Like MutateText(b []byte) ([]byte, error)?

A performance argument could be made in the reverse. The caller of AppendText could in theory spin off a goroutine to do some form of read-only validation within b[:len(b)]. Mutating that section would result in a race condition.

I think this is handled by the "Implementations must not retain b", if you make a goroutine, you are retaining b.

dsnet commented 1 year ago

So should we rename it then?

I'm arguing that we should keep it named AppendText and also keep the documentation that implementations shouldn't mutate b[:len(b)].

I think this is handled by the "Implementations must not retain b", if you make a goroutine, you are retaining b.

In my example, the asynchronous validation logic is done by the caller of AppendText, not by the implementation of AppendText.

mitar commented 1 year ago

@dsnet Thanks for the explanation!

mvdan commented 7 months ago

@ianlancetaylor could I nudge this proposal forward? It would help with the encoding/json/v2 efforts, and it seems to have reasonable consensus :)

rsc commented 6 months ago

This proposal has been added to the active column of the proposals project and will now be reviewed at the weekly proposal review meetings. — rsc for the proposal review group

rsc commented 5 months ago

Have all remaining concerns about this proposal been addressed?

The proposal is to add to package encoding:

// TextAppender is the interface implemented by an object
// that can append the textual representation of itself.
// If a type implements both [TextAppender] and [TextMarshaler],
// then v.MarshalText() must be semantically identical to v.AppendText(nil).
type TextAppender interface {
    // AppendText appends the textual representation of itself to the end of b
    // (allocating a larger slice if necessary) and returns the updated slice.
    //
    // Implementations must not retain b, nor mutate any bytes within b[:len(b)].
    AppendText(b []byte) ([]byte, error)
}

type BinaryAppender interface {
    AppendBinary([]byte) ([]byte, error)
}

And then these types would get new methods alongside their existing MarshalText and MarshalBinary methods:

go1.18.txt:pkg net/netip, method (Addr) MarshalText() ([]uint8, error)
go1.18.txt:pkg net/netip, method (AddrPort) MarshalText() ([]uint8, error)
go1.18.txt:pkg net/netip, method (Prefix) MarshalText() ([]uint8, error)
go1.2.txt:pkg encoding, type TextMarshaler interface { MarshalText }
go1.2.txt:pkg encoding, type TextMarshaler interface, MarshalText() ([]uint8, error)
go1.2.txt:pkg net, method (IP) MarshalText() ([]uint8, error)
go1.2.txt:pkg time, method (Time) MarshalText() ([]uint8, error)
go1.21.txt:pkg log/slog, method (Level) MarshalText() ([]uint8, error) #56345
go1.21.txt:pkg log/slog, method (*LevelVar) MarshalText() ([]uint8, error) #56345
go1.21.txt:pkg regexp, method (*Regexp) MarshalText() ([]uint8, error) #46159
go1.3.txt:pkg math/big, method (*Int) MarshalText() ([]uint8, error)
go1.3.txt:pkg math/big, method (*Rat) MarshalText() ([]uint8, error)
go1.6.txt:pkg math/big, method (*Float) MarshalText() ([]uint8, error)

go1.18.txt:pkg net/netip, method (Addr) MarshalBinary() ([]uint8, error)
go1.18.txt:pkg net/netip, method (AddrPort) MarshalBinary() ([]uint8, error)
go1.18.txt:pkg net/netip, method (Prefix) MarshalBinary() ([]uint8, error)
go1.2.txt:pkg encoding, type BinaryMarshaler interface { MarshalBinary }
go1.2.txt:pkg encoding, type BinaryMarshaler interface, MarshalBinary() ([]uint8, error)
go1.2.txt:pkg time, method (Time) MarshalBinary() ([]uint8, error)
go1.22.txt:pkg math/rand/v2, method (*ChaCha8) MarshalBinary() ([]uint8, error) #61716
go1.22.txt:pkg math/rand/v2, method (*PCG) MarshalBinary() ([]uint8, error) #61716
go1.8.txt:pkg net/url, method (*URL) MarshalBinary() ([]uint8, error)

Note that as long as a type has MarshalText, adding AppendText later is not a breaking change, since code should fall back to MarshalText anyway. Same for Binary. So adding these methods is not strictly speaking necessary to happen in Go 1.23 even if the new interfaces are defined in Go 1.23.

mateusz834 commented 5 months ago

And then these types would get new methods alongside their existing MarshalText and MarshalBinary methods

Probably also the crypto hashes would need AppendBinary, but only accessable through interface assertion.

EDIT: Also we need append methods for #66249.

rsc commented 5 months ago

It seems like these are a good idea, but it's a lot of new API to get right. Since the Go 1.23 freeze is next week, let's hold off on any of these until the Go 1.24 cycle.

rsc commented 5 months ago

Based on the discussion above, this proposal seems like a likely accept. — rsc for the proposal review group

The proposal is to add to package encoding:

// TextAppender is the interface implemented by an object
// that can append the textual representation of itself.
// If a type implements both [TextAppender] and [TextMarshaler],
// then v.MarshalText() must be semantically identical to v.AppendText(nil).
type TextAppender interface {
    // AppendText appends the textual representation of itself to the end of b
    // (allocating a larger slice if necessary) and returns the updated slice.
    //
    // Implementations must not retain b, nor mutate any bytes within b[:len(b)].
    AppendText(b []byte) ([]byte, error)
}

type BinaryAppender interface {
    AppendBinary([]byte) ([]byte, error)
}

And then these types would get new methods alongside their existing MarshalText and MarshalBinary methods:

go1.18.txt:pkg net/netip, method (Addr) MarshalText() ([]uint8, error)
go1.18.txt:pkg net/netip, method (AddrPort) MarshalText() ([]uint8, error)
go1.18.txt:pkg net/netip, method (Prefix) MarshalText() ([]uint8, error)
go1.2.txt:pkg encoding, type TextMarshaler interface { MarshalText }
go1.2.txt:pkg encoding, type TextMarshaler interface, MarshalText() ([]uint8, error)
go1.2.txt:pkg net, method (IP) MarshalText() ([]uint8, error)
go1.2.txt:pkg time, method (Time) MarshalText() ([]uint8, error)
go1.21.txt:pkg log/slog, method (Level) MarshalText() ([]uint8, error) #56345
go1.21.txt:pkg log/slog, method (*LevelVar) MarshalText() ([]uint8, error) #56345
go1.21.txt:pkg regexp, method (*Regexp) MarshalText() ([]uint8, error) #46159
go1.3.txt:pkg math/big, method (*Int) MarshalText() ([]uint8, error)
go1.3.txt:pkg math/big, method (*Rat) MarshalText() ([]uint8, error)
go1.6.txt:pkg math/big, method (*Float) MarshalText() ([]uint8, error)

go1.18.txt:pkg net/netip, method (Addr) MarshalBinary() ([]uint8, error)
go1.18.txt:pkg net/netip, method (AddrPort) MarshalBinary() ([]uint8, error)
go1.18.txt:pkg net/netip, method (Prefix) MarshalBinary() ([]uint8, error)
go1.2.txt:pkg encoding, type BinaryMarshaler interface { MarshalBinary }
go1.2.txt:pkg encoding, type BinaryMarshaler interface, MarshalBinary() ([]uint8, error)
go1.2.txt:pkg time, method (Time) MarshalBinary() ([]uint8, error)
go1.22.txt:pkg math/rand/v2, method (*ChaCha8) MarshalBinary() ([]uint8, error) #61716
go1.22.txt:pkg math/rand/v2, method (*PCG) MarshalBinary() ([]uint8, error) #61716
go1.8.txt:pkg net/url, method (*URL) MarshalBinary() ([]uint8, error)

Note that as long as a type has MarshalText, adding AppendText later is not a breaking change, since code should fall back to MarshalText anyway. Same for Binary. So adding these methods is not strictly speaking necessary to happen in Go 1.23 even if the new interfaces are defined in Go 1.23.

rsc commented 5 months ago

The freeze is upon us, so let's leave all the changes related to this issue for Go 1.24. Fine to mail CLs, we'll just wait to land them until the cycle opens.

rsc commented 5 months ago

No change in consensus, so accepted. 🎉 This issue now tracks the work of implementing the proposal. — rsc for the proposal review group

The proposal is to add to package encoding:

// TextAppender is the interface implemented by an object
// that can append the textual representation of itself.
// If a type implements both [TextAppender] and [TextMarshaler],
// then v.MarshalText() must be semantically identical to v.AppendText(nil).
type TextAppender interface {
    // AppendText appends the textual representation of itself to the end of b
    // (allocating a larger slice if necessary) and returns the updated slice.
    //
    // Implementations must not retain b, nor mutate any bytes within b[:len(b)].
    AppendText(b []byte) ([]byte, error)
}

type BinaryAppender interface {
    AppendBinary([]byte) ([]byte, error)
}

And then these types would get new methods alongside their existing MarshalText and MarshalBinary methods:

go1.18.txt:pkg net/netip, method (Addr) MarshalText() ([]uint8, error)
go1.18.txt:pkg net/netip, method (AddrPort) MarshalText() ([]uint8, error)
go1.18.txt:pkg net/netip, method (Prefix) MarshalText() ([]uint8, error)
go1.2.txt:pkg encoding, type TextMarshaler interface { MarshalText }
go1.2.txt:pkg encoding, type TextMarshaler interface, MarshalText() ([]uint8, error)
go1.2.txt:pkg net, method (IP) MarshalText() ([]uint8, error)
go1.2.txt:pkg time, method (Time) MarshalText() ([]uint8, error)
go1.21.txt:pkg log/slog, method (Level) MarshalText() ([]uint8, error) #56345
go1.21.txt:pkg log/slog, method (*LevelVar) MarshalText() ([]uint8, error) #56345
go1.21.txt:pkg regexp, method (*Regexp) MarshalText() ([]uint8, error) #46159
go1.3.txt:pkg math/big, method (*Int) MarshalText() ([]uint8, error)
go1.3.txt:pkg math/big, method (*Rat) MarshalText() ([]uint8, error)
go1.6.txt:pkg math/big, method (*Float) MarshalText() ([]uint8, error)

go1.18.txt:pkg net/netip, method (Addr) MarshalBinary() ([]uint8, error)
go1.18.txt:pkg net/netip, method (AddrPort) MarshalBinary() ([]uint8, error)
go1.18.txt:pkg net/netip, method (Prefix) MarshalBinary() ([]uint8, error)
go1.2.txt:pkg encoding, type BinaryMarshaler interface { MarshalBinary }
go1.2.txt:pkg encoding, type BinaryMarshaler interface, MarshalBinary() ([]uint8, error)
go1.2.txt:pkg time, method (Time) MarshalBinary() ([]uint8, error)
go1.22.txt:pkg math/rand/v2, method (*ChaCha8) MarshalBinary() ([]uint8, error) #61716
go1.22.txt:pkg math/rand/v2, method (*PCG) MarshalBinary() ([]uint8, error) #61716
go1.8.txt:pkg net/url, method (*URL) MarshalBinary() ([]uint8, error)

Note that as long as a type has MarshalText, adding AppendText later is not a breaking change, since code should fall back to MarshalText anyway. Same for Binary. So adding these methods is not strictly speaking necessary to happen in Go 1.23 even if the new interfaces are defined in Go 1.23.

mateusz834 commented 5 months ago

@rsc what about https://github.com/golang/go/issues/62384#issuecomment-2101853398? Do we need a separate proposal for these?

rsc commented 5 months ago

@mateusz834 Any unexported type that implements MarshalBinary can implement AppendBinary.

gopherbot commented 3 months ago

Change https://go.dev/cl/601595 mentions this issue: encoding: add TextAppender and BinaryAppender

gopherbot commented 3 months ago

Change https://go.dev/cl/601776 mentions this issue: crypto: implement encoding.BinaryAppender for all crypto hashes

gopherbot commented 2 months ago

Change https://go.dev/cl/603255 mentions this issue: hash: implement the encoding.BinaryAppender interface

gopherbot commented 2 months ago

Change https://go.dev/cl/603405 mentions this issue: hash: mention encoding.BinaryAppender in Hash docs

gopherbot commented 2 months ago

Change https://go.dev/cl/603815 mentions this issue: net/url: implement encoding.BinaryAppender for URL

gopherbot commented 2 months ago

Change https://go.dev/cl/605056 mentions this issue: log/slog: implement encoding.TextAppender for Level and LevelVar

gopherbot commented 2 months ago

Change https://go.dev/cl/605758 mentions this issue: math/big,regexp: implement the encoding.TextAppender interface

gopherbot commented 2 months ago

Change https://go.dev/cl/606655 mentions this issue: time: implement the encoding.(Binary|Text)Appender for Time

gopherbot commented 2 months ago

Change https://go.dev/cl/607079 mentions this issue: crypto/x509,math/rand/v2: implement the encoding.(Binary|Text)Appender

gopherbot commented 2 months ago

Change https://go.dev/cl/607520 mentions this issue: net,net/netip: implement the encoding.(Binary|Text)Appender

golang / go

encoding: add AppendText and AppendBinary #62384