mc - The Memcache Client for Node.js
mc is a correct, performant memcache client for node.js that emphasizes stability, simplicity, and flexibility over a prescriptive approach to distributed storage. The expectation is that developers handling distributed data sets will prefer to handle sharding or consistent hashing with their own logic—or may have legacy strategies in place already. One additional feature of this client is the guarantee of handling binary memcache values correctly over the simple ascii protocol. This client does not implement the new binary protocol.
Specifically, this client implements the memcache text protocol specified in the memcached project.
This library does draw inspiration from both 3rd-Eden, elbart, and ddopson: but it is a ground-up rewrite.
Stable Version: 1.0.6
Note: A major reworking to support the memcache binary protocol is underway. During this reworking, I saw what looks to be a fairly major problem in multigets across sharded clients. I would welcome a patch for that to the current 1.0.6 code base, but since I'm neck deep in the binary stuff don't intend to correct it now. If anyone is blocked on this, let me know. I could reprioritize.
The constructor takes three parameters: a server list (or just a server), an adapter, and a strategy. Adapter and Strategy functions are described more thoroughly below.
Single connection to localhost
Single connection to specified host
Connection to an array of hosts using the default CRC-hash sharding strategy
Connection to an array of hosts using a custom sharding strategy but no adapter
To disconnect from all memcache clients:
All memcache calls take as the last parameter a callback that should in turn accept error and response parameters in that order. The error param will always have a type property, and may also have a description property. Errors may be returned for ordinary conditions such as a key that is not found, or in exceptional cases, such as failure in the server connection.
Set, Add, Replace, Prepend, Append, CAS
The following snippet sets with the default options, which may be omitted. Note that the value may be a string or a buffer object. The flags option is not used by memcache; this is a user-space value of 32 bits (or 16 bits some ancient installations). The exptime is the time-to-live terminus in standard unixtime. A '0' exptime indicates no expiration. The add, replace, prepend and append methods are all identical, with the exception of the non-success results, noted below.
- STORED: Success. The result was stored.
- NOT_STORED: Failure. 'add' was attempted on an existing key, or 'replace' on a non-existent key.
- EXISTS: Failure. 'cas' was attempted on a key that had been changed since fetch.
- NOT_FOUND: Failure. 'cas' was attempted on a non-existent key. (Some servers return EXISTS in this case also.)
Get may take a single key, or an array of keys. Keys may not contain whitespace: a string with whitespace will be interpreted as multiple keys. The return result is always a map of each key to the value generated by the current adapter. The default adapter is the string adapter. The 'gets' method maps keys to an object with two properties: 'val' which maps to the results of the adapter, and 'cas' which contains the unique key.
For more on Adapters, see below.
A couple of samples.
Version will return an array of versions for each server in the memcache array.
Output might be:
All stats calls will return an array of stats values from each of the servers in the array.
Stats may also take several sub-commands. Currently documented commands are: 'slabs', 'items', and 'sizes'. These subcommands return appropriate datastructures as indicated below. Other stats variants, currently undocumented or not-yet-invented, will be parsed by the default parser.
The client may be configured with a prebuilt or a custom response adapter. The pre-built adapters are:
- mc.Adapter.string [default]
The role of an adapter is to format the results of a get or gets call for the convenience of the application. This is our least favorite part of the library today, and we are looking at a better implementation to handle adapting values on both input and output.
The raw adapter is instructive. It returns:
The application my provide any function as an adapter that operates on this raw object and returns whatever other object is suitable for the application. For example, applications might compress the value (although at this time there is no input adapter to balance the equation out, a possible future feature.)
By way of example, the implementation of the json adapter follows:
(And thus, you see, that invalid json will result in an object mapping 'val' to whatever was in memcache.)
The default sharding strategy is none for a single connection, or CRC sharding on the key for arrays of more than one. These are available for explicit selection:
Custom strategies, however, may be supplied to achieve other sharding policies. The strategy is a function taking two parameters: a key and the size of the array. It must return an integer greater than or equal to zero and less than the max value provided. E.g.:
Unit tests may be run by executing 'make test'. All methods have basic happy-case and error case coverage. Beyond unit tests, an integration test package, which expects a memcache running locally on port 11211 can be run by the command 'make integration'. These longer running tests exercise the full api in a real-world simulation including network failure scenarios. There is an even longer running app: test/simulation/runlong.js which may be used when testing more complicated network / server failure scenarios.
Pin to connman 1.0.0 & publish bug fix by maleagha
Refactor to use the connman connection management library for simpler, more robust persistent connections.
Fix for string values containing utf8 characters
Bug fix for binary values
Fix bug for gets command.
Full production testing of version 0.9.0 complete, promoted to ready-for-prime-time.
- Clustering strategies and content adapters now in beta. (See documentation, below.)
- Refactor connection and client logic.
- Rewrite tests for refactored model, better unit division.
- Result adapter model
- JSON Parser
- Simple String (Default)
- Fix integration tests for multi-get format changes.
- Documentation Complete
- Installation Ready
- Correctly handle network errors
- Protocol implementation:
- stats, incl sub-commands 'slabs', 'items', 'sizes'
- Multi-get support
- Correct binary storage/retrieval
- Test framework for unit tests
- Test framework for integration tests
- Default expiration settings