2011年12月25日 星期日

Road of Migration: From Traditional DB to Cassandra(2)

In accordance to previous post:

The following code works perfect with Jdk 7 in accompany with Cassandra 1.0.6:

May refer to this url: http://babydeed.iteye.com/category/187192


import java.nio.ByteBuffer;
import org.apache.cassandra.thrift.*;
import org.apache.cassandra.thrift.TBinaryProtocol;
import org.apache.thrift.protocol.*;
import org.apache.thrift.transport.*;

public class Test {
  public static void main(String[] args) throws Exception {
    insertOrUpdateData();
    getData();

  }

  public static void insertOrUpdateData() throws Exception{
    TTransport transport = new TFramedTransport(new TSocket("127.0.0.1",9160));
    TProtocol protocol = new TBinaryProtocol(transport);
    Cassandra.Client client = new Cassandra.Client(protocol);
    transport.open();

    client.set_keyspace("ksTest1");
    ColumnParent parent = new ColumnParent("User1");

    ByteBuffer rowid = ByteBuffer.wrap("100".getBytes());

    Column column = new Column();
    column.setName("description2".getBytes());

    column.setValue("some value here222...".getBytes());
    column.setTimestamp(System.currentTimeMillis());
    client.insert(rowid, parent, column, ConsistencyLevel.ONE);

    transport.flush();
    transport.close();


  }

  public static void getData() throws Exception{
    TTransport transport = new TFramedTransport(new TSocket("127.0.0.1",9160));
    TProtocol protocol = new TBinaryProtocol(transport);
    Cassandra.Client client = new Cassandra.Client(protocol);
    transport.open();

    client.set_keyspace("ksTest1");
    ColumnParent parent = new ColumnParent("User1");

    ByteBuffer rowid = ByteBuffer.wrap("100".getBytes());

    Column column = new Column();
    column.setName("description2".getBytes());

    ColumnPath path = new ColumnPath();
        path.column_family = "User1";
        path.column = ByteBuffer.wrap("description2".getBytes());

    ColumnOrSuperColumn c1 = client.get(rowid, path, ConsistencyLevel.ONE) ;
    String res = new String(c1.getColumn().getValue(),"UTF-8");

    System.out.println(res);
    transport.flush();
    transport.close();
  }


  public static void delete() throws Exception{
         // open connection to Cassandra server
    TTransport transport = new TFramedTransport(new TSocket("127.0.0.1",9160));
        TProtocol protocol = new TBinaryProtocol(transport);
        Cassandra.Client client = new Cassandra.Client(protocol);
        transport.open();

        // specify keyspace
        client.set_keyspace("ksTest1");

        // specify row id
        ByteBuffer rowid = ByteBuffer.wrap("100".getBytes());

        // specify column path
        ColumnPath path = new ColumnPath();
        path.column_family = "User1";
        path.column = ByteBuffer.wrap("description2".getBytes());

        // specify timestamp value of data to be removed
        long timestamp = 1301874610643000L;

        // set consistency level to one
        ConsistencyLevel consistency = ConsistencyLevel.ONE;

        // remove the description column from the specified row
        // in the User column family
        client.remove(rowid, path, timestamp, consistency);

        // release resources
        transport.flush();
        transport.close();
  }
}

Road of Migration: From Traditional DB to Cassandra(1)

As using SQL-Supported DB always encounters the tragedy of performance issue. nosql DB becomes my next of my research milestone. While figuring why all the example code posted on internet does not work. I finally figure out that the supported JAVA api is quite different from 0.6.6 to 1.0.6.

The difference is mentioned here:

If you are new to Cassandra and just want to get started, take a look at the available clients ClientOptions (v0.6) instead.

This page shows examples of using the low-level Thrift interface, primarily intended for client library developers.

To generate the bindings for a particular language, first find out if Thrift supports that language. If it does, you can run "thrift --gen XYZ interface/cassandra.thrift" for whatever XYZ you fancy.

These examples are for Cassandra 0.5 and 0.6.

As of 0.7 and later, the default thrift transport is TFramedTransport so if you get an error such as "No more data to read", switch transports.


For insertion example:

Ver 0.6.6


public void insert(java.lang.String keyspace,
                   java.lang.String key,
                   ColumnPath column_path,
                   byte[] value,
                   long timestamp,
                   ConsistencyLevel consistency_level)
            throws InvalidRequestException,
                   UnavailableException,
                   TimedOutException,
                   org.apache.thrift.TException


Ver 1.0.6

public void insert(java.nio.ByteBuffer key,
                   ColumnParent column_parent,
                   Column column,
                   ConsistencyLevel consistency_level)
            throws InvalidRequestException,
                   UnavailableException,
                   TimedOutException,
                   org.apache.thrift.TException

2011年12月19日 星期一

Oracle Golden Gate Installation On Linux

If error occurs not funding ibnnz11.so or libclntsh.so.11.1

Adding the following softlink:

# ln -s /u01/app/oracle/product/11.2.0/db_1/lib/libnnz11.so libnnz11.so
# ln -s /u01/app/oracle/product/11.2.0/db_1/lib/libclntsh.so.11.1 libclntsh.so.11.1

2011年12月10日 星期六

Recompile pyx File

Pyres is a c-extension of python, when you run into C Exception using python package. You may assume the compiled error is resulted to bug of previous version of pyrex. The best solution is to recompile the program with following command.

pyrexc .pyx -o .c

Happy Pythoning

SQLPlus 提示 Error accessing PRODUCT_USER_PROFILE

問題發生點:

當不是透過DBCA 產生資料庫時,必須手動建立系統表格

解決方案:
執行以下三支SQL
/u01/app/oracle/product/11.2.0/xe/rdbms/admin/catproc.sql
/u01/app/oracle/product/11.2.0/xe/rdbms/admin/catalog.sql
/u01/app/oracle/product/11.2.0/xe/sqlplus/admin/pupbld.sql

使用SQLPlus 出現 ERROR ORA-12514: TNS:listener does not currently know of service requested in connect descriptor

當使用SQLPlus 時, 若出現 ORA-12514 的錯誤,請檢查listener.ora 內的設定

若少了目前使用的DB Name 跟 SID Name,請添加下列數行以讓listener 辨認SID



SID_LIST_LISTENER =
  (SID_LIST =
    (SID_DESC =
      (SID_NAME = PLSExtProc)
      (ORACLE_HOME = /u01/app/oracle/product/11.2.0/xe)
      (PROGRAM = extproc)
    )
  (SID_DESC =
      (GLOBAL_DBNAME = ORCL)
      (ORACLE_HOME = /u01/app/oracle/product/11.2.0/xe)
      (SID_NAME = xe)
    )
)


設定存檔完後,使用lsnrctl reload 以重帶服務

在Ubuntu 10.4 (64bit) 上建立Oracle Instance (11.2)

少了在Windows 上 Friendly 的 Universal Installer
在Ubuntu上建立Oracle Instance 相當麻煩:

以下列出需要記住的幾點事項:
1. cp init.ora 以建立init.ora (init.ora 通常存在於$ORACLE_HOME下的dbs中)
2. 將init.ora 中的  更改為絕對路徑
3. 於init.ora建立db_create_file_dest='/u01/app/oracle/oradata'
4. 啟用Listener: lsnrctl start
5. 在SQLPlus 下輸入

CREATE DATABASE ORCL
USER SYS IDENTIFIED BY sys
USER SYSTEM IDENTIFIED BY system
EXTENT MANAGEMENT LOCAL
DEFAULT TEMPORARY TABLESPACE temp
UNDO TABLESPACE undotbs1
DEFAULT TABLESPACE users;

建立Instance ORCL

參考資料: http://docs.oracle.com/cd/B28359_01/server.111/b28310/create003.htm

2011年11月23日 星期三

AVOID PITFALL THINKING

     To solve the problem is probably everyone's first priority in confronting their work. Solving the task has nothing wrong, but people may end up stocked in a problem solving pitfall. Do before set mind on how to solve the problem. 

     Why is thinking how to solve the problem is so crucial. In my opinion, the process of problem thinking may often leads you to create a better methodology to solve the problem. And what is more, the developed methodology may be applied to same problem in the future. In that way, the created model may solve not only timely problem, but also long term issue.

     Backward a step before you leap may be a footnote of this problem solving philosophy. But it is a pity that not many people notice the importance of think before you do . Even some may adopt the methodology, how to keep the believe as central dogma will be another difficult issue. But I believe, if you put this believe into your  working attitude. Success is not far away.

2011年11月22日 星期二

BEGINNING, It's all start with a simple idea

     Years experience confronting different case help me to realize that making money from B2B selling is not difficult at all. The most important asset is to find what customers need. Requirement may vary from day-today operation solution to small but helpful tool. Both models can be developed into profitable products, but the key to develop two different types of products relies on manpower and resource.

     Indeed operational solution may make more money once your customer identifies that the solution is dispensable. However, to enter the market is hard, since most of existing leading company has already have their maintenance contractor. The meaning of selling new solution is less important than build up bounded trustworthy partnership. On the other hand, breakthrough idea made product may efface the importance of relationship. An operation independent software may be more easier for customers to buy the idea.

     In conclusion, I think for a start-up company, using innovative but operation independent solution may be helpful to enter B2B market. Once the relation bound between customer is established, killer application may find chance to enter B2B market. Then for more, turning the customer into money pot.

2011年11月19日 星期六

Estock Trade 股市分析工具

Estock Trade
Python + Django 股市分析軟體
(其實是自己的操盤工具 XD)


20111120 敏感性資料搜尋工具

Python + django 完成的B2B 軟體
敏感性資料搜尋工具



轉變成 Python 寫手

對於Scripting Language,

一直以來, 風格簡明的Python 就對我有較高的吸引力

編寫簡單、第三方套件支援性強,搭配架構簡明的django

大大提升工程師的生產力

於短時間內完成出具有競爭力的產品

尤其當要開發新模組時

類 yum install 的 easy_install 直讓我覺得自己像駭客任務的主角

Say Matplotlib, 就可以短時間內使用不同模組

看來這語言會成為我短暫做創新的生產工具之一

2011年11月18日 星期五

如何開始一個企業

在公司與客戶間遊走了許久,或許開始有資格來評論該如何創立一家公司。

就我見我問,列出下列幾點:

  1. Beginning, 人才是一切的起源 
  2. Starter, 創立公司的現金流
  3. Think Different, 如何建立與眾不同的公司
  4. Key Point, 如何建立護城河
  5. Everlasting, 長青的秘訣


sqljdbc4 與 Java 1.6.0 不相容

原本在Centos 64 位元開發的Python + Java 程式 Works 好好的 但一移到32 位元的Centos 就狀況不斷 原本以為是OS 平台的問題. 後來發現其實問題出在JVM 與JDBC 的相容性上 一旦把Jdk 跟 Jre 從 1.6.0 版升到第 7 版 一切問題迎刃而解 GOOD JOB