Shimon Tolts·Follow1 min read·Jun 14, 2017--ShareWe used Redis with a sliding window of the records sub sequence to de-duplicate data.