Clock Skew

[mark]

Overview

A few folks have identified that their Tidbyt occasionally shows the incorrect time - anywhere from a few seconds to 30 minutes in extreme cases. A clock that tells the wrong time is useless, which makes this class of bug really painful to both the Tidbyt team and our users. This post breaks down the issue and describes mitigations we’re putting in place to resolve this class of bug.

A photo of my desk, where I keep a Casio F-91W next to my Tidbyt as a last resort monitor.

The Issue

When we first started shipping Tidbyts, we used a time synchronization system that many computers use today to ensure an accurate time. As more users received their Tidbyts, we started to see numerous customer reports about Tidbyts being unable to connect.

Upon investigation, we discovered many Internet Service Providers and home networking equipment blocked or interfered with the time synchronization system we had in place. As a fix, we rolled out a less accurate time keeping system to ensure every customer who purchased a Tidbyt could connect their Tidbyt to our cloud.

The Fix

via GIPHY

We’re actively rolling out changes to our platform that will ensure Tidbyts are accurate within a few seconds while maintaining our improved connectivity. If you’re experiencing significant time skew, please feel free to reach out to [email protected] and we’ll be more than happy to investigate the issues.

Nerd Zone

For our tech savvy users, we wanted to provide a bit more detail. When we first started shipping devices, time synchronization via NTP was a required step to connect to our servers. We had a ton of issues with NTP across the board. This article provides a pretty good overview for the types of issues we were experiencing, with the most relatable being the graph showing NTP success rate for Comcast .

We switched to HTTP Date, which allowed for a much higher success rate for connections while still providing an accurate time. The issue is, after a successful connection, we relied on the real-time clock of the Tidbyt which we later discovered can skew by several minutes if left unchecked.

Our solution moving forward will be to use NTP with an HTTP Date fallback, and continuously keep the time in sync via either mechanism.

[CyberMonk]

Great writeup! Question: as someone with a custom app that currently keeps the time within ~5 seconds of accurate by brute force (AKA continuous API pushes from my own server), is this method something app developers will be able to leverage somehow? Cheers.

[rohan]

@CyberMonk The initial fix to use an alternate, better time synchronization will automatically improve all clock apps, including those developed by the community.

We’re exploring other features for further improvements — for example, allowing apps to report when they are “stale” and need to re-rendered. This is still a work in progress, but anything we roll out for our own apps does get rolled out to the broader community as well (once we’re done testing it).

[Freddicus]

I just received my Tidbyt today, and I love it! I noticed the clock skew so I came here and found this thread. Any updates?

The Tidbyt is in my living room next to my atomic clock. It’s jarring seeing my internet-connected clock be less accurate than a clock that gets its time from a signal states away. Is the firmware itself open source? Looking for contributions? Cheers!

[delfuego]

Did the solution above (" use NTP with an HTTP Date fallback, and continuously keep the time in sync via either mechanism") ever get implemented in the Tidbyt firmware? I see a mild skew in our Tidbyt clock (anywhere from 5 to 30 seconds, usually), so I came here to find out more, and greatly appreciate the detailed writeup above. Of course, it led me to packet-capture on our network to see if I’m seeing any NTP traffic at all from the device, and… I don’t think I do? I’ve let the packet capture run for 10-20 minutes at a time and never see any UDP traffic at all; I’m going to let a capture run for a few hours now, and see what I see.

[Robert_M]

New Owner of clock I two and not ammused with how in-accurate the time is I can listen to wwv and nist via phone the clock is consistantly wrong for 8 seconds or more, Any ideas how to fix I have other NTP cloce that are right on per say, and additionally I have my own gps synced NTP servers already feeding my network and devices, maybe you can add function to allow configuration of OWN ntp server so I can get accurate time on this clock. I used clockwithseconds app to see it and was odd in seconds their also appears to be a bug as Im seeing -3 -2 -1 etc in seconds display on tidbyt itself app on ipad not showing that. Im replace my many BRG clocks as they fail and way tring yours as seems to be a reasonable cost, but because time is not accurate I might need to reconsider. a clock thats wrong is useless. so to test yourself, just use clock withseconds app I have it Pinned at moment, and call NIST is best choice I have navy as well both same about 8 seconds off
NISt voice number is 303 499 7111 it will auto disconnect after 2 mins or so .