Word16 FromField and ToField instances

[Pitometsu]

Hi there!

I'm trying to store Word16 type into char(2) field, bit-to-bit.

So, here's what I have:

newtype UInt16 = UInt16 { fromUInt16 :: Word16 } -- new type for new instances

instance ToField UInt16 where
  toField = Plain . word16LE . fromUInt16  -- write data in little-endian
  {-# INLINE toField #-}

Now I need FromField to read data. What I see in Database.PostgreSQL.Simple.FromField is

-- | \"char\", bpchar
instance FromField Char where
    fromField f bs =
        if $(mkCompats [TI.char,TI.bpchar]) (typeOid f)
        then case bs of
               Nothing -> returnError UnexpectedNull f ""
               Just bs -> if B.length bs /= 1
                          then returnError ConversionFailed f "length not 1"
                          else return $! (B.head bs)
        else returnError Incompatible f ""

-- | int2
instance FromField Int16 where
    fromField = atto ok16 $ signed decimal

-- | bytea, name, text, \"char\", bpchar, varchar, unknown
instance FromField SB.ByteString where
    fromField f dat = if typeOid f == $(inlineTypoid TI.bytea)
                      then unBinary <$> fromField f dat
                      else doFromField f okText' pure dat

-- | bytea
instance FromField (Binary SB.ByteString) where
    fromField f dat = case format f of
      PQ.Text   -> doFromField f okBinary (unescapeBytea f) dat
      PQ.Binary -> doFromField f okBinary (pure . Binary) dat

I'm not so good in TH, so have no clue how to read some parts of this code. Could you suggest, please, how can I implement FromField instance without TH?

[Pitometsu]

Would be correct something like:

instance FromField UInt8 where
  fromField f mbs = case mbs of
    Nothing -> returnError UnexpectedNull f ""
    Just bs -> case runGet getWord8 bs of
      Left errorMessage -> returnError ConversionFailed f errorMessage
      Right result      -> return $! UInt8 $! result

instance FromField UInt16 where
  fromField f mbs = case mbs of
    Nothing -> returnError UnexpectedNull f ""
    Just bs -> case runGet getWord16le bs of
      Left errorMessage -> returnError ConversionFailed f errorMessage
      Right result      -> return $! UInt16 $! result

instance FromField UInt32 where
  fromField f mbs = case mbs of
    Nothing -> returnError UnexpectedNull f ""
    Just bs -> case runGet getWord32le bs of
      Left errorMessage -> returnError ConversionFailed f errorMessage
      Right result      -> return $! UInt32 $! result

instance FromField UInt64 where
  fromField f mbs = case mbs of
    Nothing -> returnError UnexpectedNull f ""
    Just bs -> case runGet getWord64le bs of
      Left errorMessage -> returnError ConversionFailed f errorMessage
      Right result      -> return $! UInt64 $! result

[lpsmith]

You would probably be better off using thebytea type to store a Word16... either that or create your own type (probably as a server side c extension) I don't think char allows the ascii null character, so Word16 would have 511 unrepresentable values.

I am not the best with TH either. All inlineTypoid does is select the typoid field from a TypeInfo record, but at compile time. At least at one point GHC was having difficulty doing that at compile time without TH. mkCompats simply checks if an expression is equal to any one of the typoid fields in a selection of TypeInfo records. You can mentally ignore the $( ) and pretend they are regular functions.

Right now, your functions should basically work, though they don't perform any type checking.

[Pitometsu]

@lpsmith thank you for suggestions, for now it looks like this:

newtype UInt8 =
  UInt8
    { fromUInt8  :: Word8
    } deriving (Generic, Eq, Ord, Show, Num, Real, Enum, Integral)

newtype UInt16 =
  UInt16
    { fromUInt16 :: Word16
    } deriving (Generic, Eq, Ord, Show, Num, Real, Enum, Integral)

newtype UInt32 =
  UInt32
    { fromUInt32 :: Word32
    } deriving (Generic, Eq, Ord, Show, Num, Real, Enum, Integral)

newtype UInt64 =
  UInt64
    { fromUInt64 :: Word64
    } deriving (Generic, Eq, Ord, Show, Num, Real, Enum, Integral)

--------------------------------------------------------------------------------

instance ToField UInt8 where
  {-# INLINE toField #-}
  toField = Plain . convert . fromUInt8
    where
      convert w = int8 $ fromRight $ runGet getInt8 $ runPut $ putWord8 w

instance ToField UInt16 where
  {-# INLINE toField #-}
  toField = Plain . int16LE . convert . fromUInt16
    where
      convert w = fromRight $ runGet getInt16le $ runPut $ putWord16le w

instance ToField UInt32 where
  {-# INLINE toField #-}
  toField = Plain . int32LE . convert . fromUInt32
    where
      convert w = fromRight $ runGet getInt32le $ runPut $ putWord32le w

instance ToField UInt64 where
  {-# INLINE toField #-}
  toField = Plain . int64LE . convert . fromUInt64
    where
      convert w = fromRight $ runGet getInt64le $ runPut $ putWord64le w

--------------------------------------------------------------------------------

-- | not implemented in pg
instance FromField UInt8 where
  fromField f mbs = if typeOid f /= undefined
    then returnError Incompatible f "Don't know 1 byte integer type in Postgres"
    else case mbs of
      Nothing -> returnError UnexpectedNull f ""
      Just bs -> case runGet getInt8 bs of
        Left errorMessage -> returnError ConversionFailed f errorMessage
        Right result      -> return $! UInt8 $! convert result
          where
            convert i = fromRight $ runGet getWord8 $ runPut $ putInt8 i

-- | smallint
instance FromField UInt16 where
  fromField f mbs = if typeOid f /= typoid TI.int2
    then returnError Incompatible f "Should be smallint value"
    else case mbs of
      Nothing -> returnError UnexpectedNull f ""
      Just bs -> case runGet getInt16le bs of
        Left errorMessage -> returnError ConversionFailed f errorMessage
        Right result      -> return $! UInt16 $! convert result
          where
            convert i = fromRight $ runGet getWord16le $ runPut $ putInt16le i

-- | integer
instance FromField UInt32 where
  fromField f mbs = if typeOid f /= typoid TI.int4
     then returnError Incompatible f "Should be integer value"
     else case mbs of
       Nothing -> returnError UnexpectedNull f ""
       Just bs -> case runGet getInt32le bs of
         Left errorMessage -> returnError ConversionFailed f errorMessage
         Right result      -> return $! UInt32 $! convert result
           where
             convert i = fromRight $ runGet getWord32le $ runPut $ putInt32le i

-- | bigint
instance FromField UInt64 where
  fromField f mbs = if typeOid f /= typoid TI.int8
    then returnError Incompatible f "Should be bigint value"
    else case mbs of
      Nothing -> returnError UnexpectedNull f ""
      Just bs -> case runGet getInt64le bs of
        Left errorMessage -> returnError ConversionFailed f errorMessage
        Right result      -> return $! UInt64 $! convert result
          where
            convert i = fromRight $ runGet getWord64le $ runPut $ putInt64le i

[lpsmith]

Converting unsigned integer types to signed integer types like this is a possibility, but it is very ugly: now you can operate on these types server-side, but in ways that don't make sense. And it's not clear from just looking at the schema to have some idea of what these values actually mean. And, there's no type checking between signed and unsigned types, so it would be easy to make signed-versus-unsigned errors.

If it's an option to you, writing a C server-side extension for unsigned integer types is surprisingly easy, check out the complex number example include in the source distribution. And, there's probably already somebody who has published an open-source extension to do exactly this; you might check github and pgfoundry.

If custom extensions to your postgres deployment target are impractical or impossible, I would strongly suggest using bytea to store all the unsigned word types; it'll add one byte to the size of the value to store the length of the value, but it does allow for storing arbitrary binary bit patterns with full fidelity. And, because the textual input syntax is hexadecimal, and the textual output syntax is binary, it's likely to be significantly faster than the decimal-based textual input/output syntax for regular signed integers. And, a bytea type is relatively opaque in postgresql, you could concatinate two unsigned integer types server-side, which is unlikely to make much sense, but you'd avoid possible sign errors involved in other computations, and make it clear-ish to those examining the schema directly that this is some kind of relatively-opaque data.

In this case, you would need to generate array syntax, probably the easiest clean way to do that would be to leverage what's already in postgresql-simple, e.g. by writing

instance ToField UInt16 where
  {-# INLINE toField #-}
  toField = toField . convert . fromUInt16
    where
       convert :: Word16 -> Binary
       convert = ...

[lpsmith]

This looks promising: https://github.com/petere/pguint

[lpsmith]

If you want to use that extension, it would be best to not use the typeOid to perform type checking, because the typoid of the uint types change from database to database. Rather, use the typename operator, and do a string comparison.