Deserializing optional date times with Serde

April 30th, 2019

As part of the CLI app I mentioned previously I used Serde to parse a JSON response from a HTTP API. The response has an optional date time value with a custom format. It took a little while for me to figure out how to handle it so I thought I’d write it up here.

The problem

The response has a last_updated field that is either not present or a UTC date time in a custom format. The chrono crate handles date times with the DateTime<Utc> type. As the date time value might not be present in the response the type in the struct is Option<DateTime<Utc>>.

The struct looks like:

#[derive(Deserialize, Debug)]
struct DataResponse {
    error_message: Option<String>,
    #[serde(default)]
    #[serde(deserialize_with = "deserializer::deserialize_optional_datetime")]
    last_updated: Option<DateTime<Utc>>,
    rows: Vec<DataPoint>,
    status: DataResponseStatus,
}

The dates look like 07-Mar-2019 21:00:00. There is another field in the response that is always present with the same date time format.

The solution

Using deserialize_with tells Serde to use custom deserialization code for the last_updated field. The deserialize_optional_datetime code is:

pub fn deserialize_optional_datetime<'de, D>(d: D) -> Result<Option<DateTime<Utc>>, D::Error>
where
    D: de::Deserializer<'de>,
{
    d.deserialize_option(OptionalDateTimeFromCustomFormatVisitor)
}

The types should be familiar if you’ve used Serde before. If you’re not used to Rust then the function signature will likely look a bit strange. What it’s saying is that d will be something that implements Serde’s Deserializer trait, and that any references to memory will live for the 'de lifetime.

You don’t need to understand all this fully before you can use Serde usefully though it will help when you want to do more advanced serialization and deserialization.

The code calls d.deserialize_option passing in a struct. The passed in struct implements Serde’s Visitor trait and looks like:

struct OptionalDateTimeFromCustomFormatVisitor;

impl<'de> de::Visitor<'de> for OptionalDateTimeFromCustomFormatVisitor {
    type Value = Option<DateTime<Utc>>;

    fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
        write!(formatter, "null or a datetime string")
    }

    fn visit_none<E>(self) -> Result<Self::Value, E>
    where
        E: de::Error,
    {
        Ok(None)
    }

    fn visit_some<D>(self, d: D) -> Result<Option<DateTime<Utc>>, D::Error>
    where
        D: de::Deserializer<'de>,
    {
        Ok(Some(d.deserialize_str(DateTimeFromCustomFormatVisitor)?))
    }
}

The struct doesn’t have any fields, it just a type we can use to implement the custom visitor. Serde’s Visitor trait has default implementations for all it’s visit_* functions so only visit_none and visit_some need to be implemented.

visit_none returns Ok(None) so the last_updated value in the struct will be None.

visit_some delegates the parsing to another custom deserializer via the deserialize_str call. It looks like:

struct DateTimeFromCustomFormatVisitor;

pub fn deserialize<'de, D>(d: D) -> Result<DateTime<Utc>, D::Error>
where
    D: de::Deserializer<'de>,
{
    d.deserialize_str(DateTimeFromCustomFormatVisitor)
}

impl<'de> de::Visitor<'de> for DateTimeFromCustomFormatVisitor {
    type Value = DateTime<Utc>;

    fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
        write!(formatter, "a datetime string")
    }

    fn visit_str<E>(self, value: &str) -> Result<Self::Value, E>
    where
        E: de::Error,
    {
        match NaiveDateTime::parse_from_str(value, "%d-%b-%Y %H:%M:%S") {
            Ok(ndt) => Ok(DateTime::from_utc(ndt, Utc)),
            Err(e) => Err(E::custom(format!("Parse error {} for {}", e, value))),
        }
    }
}

The main part is in visit_str which parses the value using NaiveDateTime::parse_from_str from chrono. If it parses successfully it’s converted into a DateTime<Utc> or the Err will get propagated back up to Serde and reported.

Summary

Serde has worked well for me for what I’ve used it for so far. This is the first time I’ve needed some customisation and it took me a little bit of time to understand what I needed to do.

The visitor pattern is a well established way of handling code like this and makes a lot of sense here. For a relatively new Rust programmer there are a lot of types here but once you get your head round them it’s possible to see exactly what they’re specifying and how they work.