Redis: LPUSH Command

Redis is a key value store. I recently used it in multiple projects and quite impressed by its simplicity and performance. it can be used for caching and storing simple nosql data.

In my later blogs, I will explain how to use it as reliable queue using RPOPLPUSH command. Before looking at that lets look at LPUSH command. LPUSH command works on Redis list data structure and it inserts an item at the tail of the list. Following shows it nicely:

REDIS  LPUSH COMMAND

Promote your blog

HBase Secondary Index: Scaning Using Multiple Keys

In HBase one can define row keys to store primary key and do scan using partial or full row keys as explained in previous blogs but what if we need another dimension to do search? That is we may need do a look up based on some other field which is not in our primary table row key.

 To achieve this we need to define a second table where row key have this second dimension and it stores the value of the primary row key. Lets look at the example of my previous blog, where we are storing sessions of users.

  userIdBytes+seperatorByte+dateStringByte+seperatorByte+sessionIdBytes

Here seperatorByte should be choosen in such a way which does not conflict with userId and sessionId bytes values. For example use LF (decimal 10)

Lets call above key as primaryKey.

Note that using above row key we can store, column and values in one or more column families for each
user session row. For example we may want to store session startTime and endTime as two values in two columns in a column family.

 Now lets say that we want to get all user sessions in a given geographical area (Get all users in US or get all users in UK), so how do we store this information? To do this can define a second table where row key is geographical area code plus primaryRow Key (ie. userId date and sessionId.)

geoAreaCodeBytes+seperatorByte+primaryKey

Now we can do a search for user sessions using geographical area, similar to what I explained in my previous blog. Also for each of these rows, we can find the primaryKey row for this row key itself and use it to find additional information that we store for user sessions in primaryKey.

ActiveMQ Network Of Brokers (Network Connector) Vs Network Bridge

Network Connector:

ActiveMQ network of brokers configuration is achieved using network connector. A network connector specifies one or more ActiveMQ servers/brokers where messages are forwarded or pushed from the ActiveMQ server which has this network connector is configured.

Following diagram makes it clear:

There is a duplex property on network connector configuration which allows bi-directional message flow similar to network bridge. Wth this property set to true, all destinations configured in network connector will be available for bi-directional message flow, where as in network bridge one can choose explicit destinations. For example you want to forward all the messages from Hub to spoke but only want to forward selective destination message from spoke to hub then a network bridge is better.

Network Bridge:
A network bridge configuration on a ActiveMQ server/broker, allows message to be selectively consumed from other ActiveMQ brokers or forwarded from this source ActiveMQ server/broker to other ActiveMQ server/broker.

Following diagram makes it clear:

Promote your blog

Spring enable @Transactional annotation

In Spring to enable @Transactional Annotation processing you would need to specify  <tx:annotation-driven>

in your Spring xml configuration file.

Example:

<tx:annotation-driven transaction-manager="txManager"> </tx:annotation-driven> Promote your blog

HBase Composite Row Key Design: Doing Table Scan Using Partial Row Key Part2

In my previous blog I described how to design HBase table row key which allows partial row key scan. Now in this second blog lets look at another example of composite row key where we will have 3 seperate fields in the key.

The key will consist of userId, date and sessionId. Something like this:

userIdBytes+seperatorByte+dateStringByte+seperatorByte+sessionIdBytes
Here seperatorByte should be choosen in such a way which does not conflict with userId and sessionId bytes values. For example use LF (decimal 10)


This key design allows partial key scan in following ways:

• To find all the sessions for a given user just specify  userIdBytes as start row when creating table Scan object.
• To find all the session for a given user on a given date or a given range of date  just specify  userIdBytes+seperatorByte+dateStringByte  as start row and end row, when creating a table Scan object.
• To find a specific sessionId, specify whole key userIdBytes+seperatorByte+dateStringByte+seperatorByte+sessionIdBytes, as start row, when creating table Scan object.

So we can see the key design facilitates two different ways of doing a partial scan on the HBase table.




Promote your blog

HBase Composite Row Key Design: Doing Table Scan Using Partial Row Key

HBase is a column oriented NoSql Database which uses Hadoop file system as the data storage.The rows in HBase are stored in sorted order, meaning they are alphabetically sorted.Typically one creates an HTable with one or more column family and store data in it.

A column family is a collection of dynamic columns, meaning column name can be defined when storing data and so one can have n number of columns and there is no limit to it.Data in a table is stored using a row key and specifying column family, column name and column value. The row key is just a byte array.


The row key of a HBase table can be a composite key, consisting of multiple individual keys.For example one can design a key to keep track of user and number of session a user has.In this case a row key can be:


userIdBytes+seperatorByte+sessionIdBytes.


Here seperatorByte should be choosen in such a way which does not conflict with userId and sessionId bytes values. For example use LF (decimal 10)

This row key design allows partial table scanning, where to get all the sessionIds for a user one can simple create a HBase Scan object with userId as the start row of the Scan and get all table rows for this userId. Meaning one can get all the rows which start with this userId.

The other advantage of this key design is that one can extract sessionId from the row key itself without actually loading data from HBase.




Promote your blog

example of PKCS7Padding and ISO10126Padding

In my previous blog about How Padding Affects Block Cipher Encryption: PKCS7Padding vs ISO10126Padding, I explained why using the correct padding is important when using block cipher. Here is an example which demonstrate what happens, if we use the wrong padding for decryption.

The case1 demonstrates what I explained in previous blog, Here we encrypt password using PKCS7Padding and decrypt using ISO10126Padding and it works fine.


The case2 demonstrate the case where we encrypt password using ISO10126Padding and
decrpyt using PKCS7Padding and this fails with an error like:

javax.crypto.BadPaddingException: pad block corrupted decrypted password with PKCS7Padding: null

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.io.OutputStream;
import java.security.Security;
import javax.crypto.Cipher;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import com.sun.xml.internal.messaging.saaj.packaging.mime.internet.MimeUtility;

public class EncryptionPaddingTest {

    private static final String KEY = "myowntestkey01234567890123456789";

    private static final String vector = "myowntestvector!";

    public static void main(String[] args) {
        Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider());
 
        String password = "1234";
 
        //Case1: encrypt password using PKCS7Padding padding and decrypt using ISO10126Padding padding
        doEncryptionDecryption(password, "PKCS7Padding", "ISO10126Padding");
 
        //Case2: encrypt password using ISO10126Padding padding and decrypt using PKCS7Padding padding
        doEncryptionDecryption(password, "ISO10126Padding", "PKCS7Padding");
 
    }

    public static void doEncryptionDecryption(String input, String encryptionPadding, String decryptionPadding) {
 
        String encodedPassword = encryptUsingAES(input, encryptionPadding);
        System.out.println("encrypted password with " + encryptionPadding +": " + encodedPassword);
 
        //(2) decrypted password using encryptionPadding padding
        String decodedPassword1 = decryptUsingAES(encodedPassword, encryptionPadding);
        System.out.println("decrypted password with " +encryptionPadding + ": "+ decodedPassword1);
 
        //(3) decrypted password using decryptionPadding padding
        String decodedPassword2 = decryptUsingAES(encodedPassword, decryptionPadding);
        System.out.println("decrypted password with " +decryptionPadding + ": "+ decodedPassword2);
 
    }

    public static String encryptUsingAES(String input, String padding) {
        try {
            byte[] bytes = input.getBytes("UTF-8");
            Cipher cipher = Cipher.getInstance("Rijndael/CBC/"+padding);
            SecretKeySpec keySpec = new SecretKeySpec(KEY.getBytes(), "Rijndael");
            cipher.init(Cipher.ENCRYPT_MODE, keySpec, new IvParameterSpec(vector.getBytes()));
            byte[] encrypted = cipher.doFinal(bytes);
            byte[] base64Encoded = encodeBase64(encrypted);
            return new String(base64Encoded, "UTF-8");
            } catch (Exception e) {
            e.printStackTrace();
        }
 
        return null;
    }


    public static String decryptUsingAES(String input, String padding) {
        try {
            byte[] bytes = input.getBytes("UTF-8");
            byte[] base64Decoded = decodeBase64(bytes);
            Cipher cipher = Cipher.getInstance("Rijndael/CBC/"+padding);
            SecretKeySpec keySpec = new SecretKeySpec(KEY.getBytes(), "Rijndael");
            cipher.init(Cipher.DECRYPT_MODE, keySpec, new IvParameterSpec(vector.getBytes()));
            byte[] decrypted = cipher.doFinal(base64Decoded);
     
            return new String(decrypted, "UTF-8");
            } catch (Exception e) {
            e.printStackTrace();
        }
 
        return null;
    }


    private static byte[] encodeBase64(byte[] b) throws Exception {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        OutputStream b64os = MimeUtility.encode(baos, "base64");
        b64os.write(b);
        b64os.close();
        return baos.toByteArray();
    }

    private static byte[] decodeBase64(byte[] b) throws Exception {
        ByteArrayInputStream bais = new ByteArrayInputStream(b);
        InputStream b64is = MimeUtility.decode(bais, "base64");
        byte[] tmp = new byte[b.length];
        int n = b64is.read(tmp);
        byte[] res = new byte[n];
        System.arraycopy(tmp, 0, res, 0, n);
        return res;
    }

}