Image for post
Image for post

Suppose our Users service cluster has 4 nodes and each respond to client requests behind a load balancing system. Each node also has a cache which should be consistent with the Users database. So how to update each node’s cache if users database has changed? This story introduces a resource-version-keeping approach which may be more economical.

Possible Solutions

There are two obvious solutions, while both have pros and cons.

Update Cache Periodically

That is, we use a ticker to fetch data from database and reset cache on an interval basis. Easy as it is, it’s expensive in that it calls database every time, which is unnecessary in most cases since the database may not change at all. …


Image for post
Image for post

If you often need to process data from Message Queue Systems like Kafka, you may often wonder how to consume data efficiently. Specifically, how could you process 1 billion or more messages within an hour? Then this story is just for you.

In this story, I proposed four consuming patterns using the well-known golang package Sarama and performed several experiments to show how different patterns affect the final performance. Let’s go.

Setup

To do this test, I established a simple cluster with three nodes, one for Kafka broker, one for the producer server and the final as the consumer server, as shown in Fig…


Image for post
Image for post

When we read golang source code or learn to use some builtin structs, frequently we may be told that “must not be copied after first use”, such as sync.Cond, sync.Map, sync.Mutex (nearly all types in sync package) and strings.Builder. Most of the time it is required so for safety reasons, for example you have a struct with a pointer field and you don’t want it to be copied since a shallow copy will make these two hold the same pointer and be unsafe. So what does golang do to ensure that? …


One word for anyone who doesn’t know about the filter yet: a simple but strong algorithm which combines estimates and measurements to give better predications of some linear systems.

I learned it when studying self-driving during which I revisited really a lot of knowledge of linear algebra. So I wrote my notes here and hope that it would be your most easy-to-understand kalman filter primer.

First of all, let’s assume a linear system which is modelled by the following two equations:

Image for post
Image for post
Fig 1

where

  • Xt (n*1) is the state vector of a process at time t, e.g., …


Image for post
Image for post

First of all, one word for anyone who does not know about plupload: a popular Javascript library that handles nearly all of file uploading matters.

Recently I came across a weird issue: users uploaded files to our server using plupload but got a series of 408 responses with zero chunk size for each request. I divided this into two sub-issues:

  1. why got 408
  2. why zero chunk size

Then I read the source code of Plupload and tried to find the retry logic. The code that are relevant to error-retry:

function handleError() {  if (retries-- > 0) {    delay(uploadNextChunk, 1000);  } else


For developers who still use Angular 1.x just like me.

Frequently we may come across the situation that we filter some array and then want to use the filtered items in our controller, like the following:

// html
<div ng-repeat="book in filteredBooks = (books | customFilter:'xx')">
...
</div>
// controller
$scope.books = ['Ajax', 'Html', 'CSS']
$scope.filteredBooks ...

Unfortunately this usage will result in ‘undefined’ for the ‘filteredBooks’ variable, though it works for filtering the books. I googled a lot and tried several solutions like here, but still not worked and seemed like missing the point. …


Recently during my NG development, I solved a performance issue with the help of Chrome profiling tool, and would like to share and record the process here.

This is my company internal product and still using Angular 1.x (no intention to explain why still 1.x, all in all, it’s enough). Generally it runs by fetching data from backend and then rendering some lists or charts for operation use. And my job is just to show the data using two different views. While since the two views are both lists(you may simply think that they are the same array ordered by different properties, but more than that), so I use the same template. …


Great product. Simply hope it could live longer than I can imagine.

About

Jing

Breathtaking interfaces and strong services together make great products

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store