overclocked

Exceeding the Manufacturer's Specifications

Contents

Memcache Client

Introduction

Status

Install

Usage

Creation

Connection

... to localhost
... to specified host
... to an array of hosts
.. without an adapter

Disconnect

Error Responses

Write Operations

Set, CAS, etc.
Increment, Decrement

Read Operations

Get, Gets

Miscellaneous

Version
Stats
Subcommands

Adapters

Strategies

Testing

Change Log

mc - The Memcache Client for Node.js

Introduction

mc is a correct, performant memcache client for node.js that emphasizes stability, simplicity, and flexibility over a prescriptive approach to distributed storage. The expectation is that developers handling distributed data sets will prefer to handle sharding or consistent hashing with their own logic—or may have legacy strategies in place already. One additional feature of this client is the guarantee of handling binary memcache values correctly over the simple ascii protocol. This client does not implement the new binary protocol.

Specifically, this client implements the memcache text protocol specified in the memcached project.

This library does draw inspiration from both 3rd-Eden, elbart, and ddopson: but it is a ground-up rewrite.

See also: the annotated source code, the license, the full source code, or the the NPM page.

Status

Stable Version: 1.0.6

Note: A major reworking to support the memcache binary protocol is underway. During this reworking, I saw what looks to be a fairly major problem in multigets across sharded clients. I would welcome a patch for that to the current 1.0.6 code base, but since I'm neck deep in the binary stuff don't intend to correct it now. If anyone is blocked on this, let me know. I could reprioritize.

Install

npm install mc

Usage

Creation

The constructor takes three parameters: a server list (or just a server), an adapter, and a strategy. Adapter and Strategy functions are described more thoroughly below.

Connection

Single connection to localhost
Single connection to specified host
Connection to an array of hosts using the default CRC-hash sharding strategy
Connection to an array of hosts using a custom sharding strategy but no adapter

Disconnect

To disconnect from all memcache clients:

Error Responses

All memcache calls take as the last parameter a callback that should in turn accept error and response parameters in that order. The error param will always have a type property, and may also have a description property. Errors may be returned for ordinary conditions such as a key that is not found, or in exceptional cases, such as failure in the server connection.

Write Operations

Set, Add, Replace, Prepend, Append, CAS

The following snippet sets with the default options, which may be omitted. Note that the value may be a string or a buffer object. The flags option is not used by memcache; this is a user-space value of 32 bits (or 16 bits some ancient installations). The exptime is the time-to-live terminus in standard unixtime. A '0' exptime indicates no expiration. The add, replace, prepend and append methods are all identical, with the exception of the non-success results, noted below.

Success Response:

Failure Responses:

Increment, Decrement

The value parameter in these methods is optional, and defaults to 1. Possible error types: NOT_FOUND, when the key does not exist; CLIENT_ERROR, when the value is not numeric.

Read Operations

Get, Gets

Get may take a single key, or an array of keys. Keys may not contain whitespace: a string with whitespace will be interpreted as multiple keys. The return result is always a map of each key to the value generated by the current adapter. The default adapter is the string adapter. The 'gets' method maps keys to an object with two properties: 'val' which maps to the results of the adapter, and 'cas' which contains the unique key.

For more on Adapters, see below.

A couple of samples.

Miscellaneous

Version

Version will return an array of versions for each server in the memcache array.

Output might be:

[ '1.4.5', '1.4.5', '1.4.6' ]
Stats

All stats calls will return an array of stats values from each of the servers in the array.

Stats may also take several sub-commands. Currently documented commands are: 'slabs', 'items', and 'sizes'. These subcommands return appropriate datastructures as indicated below. Other stats variants, currently undocumented or not-yet-invented, will be parsed by the default parser.

Stats Subcommands

Adapters

The client may be configured with a prebuilt or a custom response adapter. The pre-built adapters are:

The role of an adapter is to format the results of a get or gets call for the convenience of the application. This is our least favorite part of the library today, and we are looking at a better implementation to handle adapting values on both input and output.

The raw adapter is instructive. It returns:

{ buffer: size: flags: key: cas: }

The application my provide any function as an adapter that operates on this raw object and returns whatever other object is suitable for the application. For example, applications might compress the value (although at this time there is no input adapter to balance the equation out, a possible future feature.)

By way of example, the implementation of the json adapter follows:

(And thus, you see, that invalid json will result in an object mapping 'val' to whatever was in memcache.)

Strategies

The default sharding strategy is none for a single connection, or CRC sharding on the key for arrays of more than one. These are available for explicit selection:

Custom strategies, however, may be supplied to achieve other sharding policies. The strategy is a function taking two parameters: a key and the size of the array. It must return an integer greater than or equal to zero and less than the max value provided. E.g.:

Testing

Unit tests may be run by executing 'make test'. All methods have basic happy-case and error case coverage. Beyond unit tests, an integration test package, which expects a memcache running locally on port 11211 can be run by the command 'make integration'. These longer running tests exercise the full api in a real-world simulation including network failure scenarios. There is an even longer running app: test/simulation/runlong.js which may be used when testing more complicated network / server failure scenarios.

Change Log

v 1.0.6

Pin to connman 1.0.0 & publish bug fix by maleagha

v 1.0.5

Refactor to use the connman connection management library for simpler, more robust persistent connections.

v 1.0.4

Fix for string values containing utf8 characters

v 1.0.3

Documentation overhaul

v 1.0.2

Bug fix for binary values

v 1.0.1

Fix bug for gets command.

v 1.0.0

Full production testing of version 0.9.0 complete, promoted to ready-for-prime-time.

v 0.9.0

v 0.8.0

v 0.6.1

v 0.6.0