Thursday, June 9, 2011

Turn back to Bochs

After a few days tried to use QEMU as an emulator for studying low-level software, it turned out to me that QEMU doesn't support much for debugging. Things like step-by-step run, set break points,... is unable. Then I went back to have a try with Bochs. It's amazing that Bochs is not that hard as I've ever thought. It even provides very good debugging features.

Here're some of my notes on Bochs installation and use:

  1. Download Bochs source code at http://bochs.sourceforge.net/getcurrent.html

  2. Extract Bochs code
      $ gunzip -c bochs-version.tar.gz | tar -xvf -

  3. Configure to use debugger
      $ ./configure --enable-debugger --enable-disasm

  4. Install
    $ make
    $ sudo make install

  5. There an example Bochs configs file name .bochsrc at extracted dir

Sunday, May 29, 2011

Intel vs AT&T syntax

There're two popular assembly syntaxes: Intel and AT&T. Intel syntax is popular in Windows world. In Linux, AT&T syntax is more popular though GAS (Gnu Assembler) supports both.

The following lists some major differences of the two syntaxes:

  1. AT&T prefixes register with % sign
    * Intel:
       eax, ebx, ecx,...
    * AT&T:
       prefix by % sign: %eax, %ebx, %ecx,...

  2. AT&T prefixes immediate value with $ sign, Intel is not
    * Intel: 10, 80h
    * AT&T: $10, $0x80

  3. AT&T and Intel syntax use opposite instruction operands
    * Intel:      mnemonic destination, source 
       Ex:         mov eax, 100
    * AT&T:  mnemonic source, destination
       Ex:        movl $100, %eax

  4. AT&T suffixes instruction to specify instruction's operand size (1 byte: b, 2 bytes: w, 4 bytes: l) but Intel uses directive before operands (1 byte: byte ptr, 2 bytes: word ptr, 4 bytes: dword ptr)
    * Intel:
       mov al, byte ptr foo
    * AT&T:
       movb foo, %al

Saturday, May 28, 2011

Assembly Language

Assembly can be seen as machine language but in symbols/mnemonics instead of 0s or 1s. So one can make use of any aspect of computer's power if writing programs in assembly.

Assembly is specific to machine architecture. IA32 (also called x86, i386) is the most popular architecture for PC.
  1. Register
    Can be classified in 4 types:
    - general purpose (eax, ebx, ecx, edx)
    - pointer/index (esp, ebp, esi, edi)
    - instruction pointer (eip)
    - flags (eflags)

    These are all 32 bits. Each register contains 8-bit and 16-bit parts. Ex: eax (32 bits), ax (16 bits), ah (8 bits), al (8bits).

  2. Instruction
    - arithmetic/logic: add, sub, and, or,...
    - control: jmp,..
    - data movement: mov,..

  3. Operand
    - register: operand value is contained in register
    - immediate: operand value is a constant
    - memory: operand value is in memory

  4. Addressing mode
    Addressing mode is the way to specify a memory address.
    - absolute:
      address = a value
    - register:
      address = register content
    - displacement:
      address = register content + a value
    - indexed:
      address = register content + a value + another register content (index) * another value (scale)

  5. Subroutine
    - subroutine is a set of instructions
    - parameters passed to subroutine are usually pushed on stack

Tuesday, May 17, 2011

Some notes on QEMU

I am finding a computer emulator so that I can use to experiment some low-level softwares. There are two prominent free and open-source emulators: Bochs and QEMU. Bochs seems more popular but lacks of documentation. So I have decided to use QEMU because it's quite simple to use and well documented.

1. Installation (on Ubuntu)
    $ sudo apt-get install qemu

2. Start emulator
    $ qemu [options] [disk_image] 
       --> This means start an emulator with specified options and disk_image (usually contains OS)

    Ex:
      $ qemu linux.img
             --> Start an emulator with default options and its hard disk contains linux.img

    There are lots of options which specify how your emulated computer could be such as what type of its cpu, hard disk, video card, sound card,... You can get more details on each option in QEMU documentation.

3. Monitoring
    QEMU provides a way to monitoring your emulator in which you can inspect your emulator, control it, change its devices, query its status,...

    You can switch back and forth between the emulator and its monitor with keystrokes: Ctrl+Alt+2 and Ctrl+Alt+1

    Some monitoring commands:
    (qemu) help or ? [cmd] 
    (qemu) change device setting
    (qemu) x/fmt addr
           Virtual memory dump starting at addr
    (qemu) xp/fmt addr
           Physical memory dump starting at addr 
    ...


Monday, May 16, 2011

Use helpers in controller

Rails applied Restful as part of its design.

To follow Restful, you should:

1. Model your web app as resources
2. Manipulate your app's resources through a conventional interface

Ex:
- if you have Product resource then urls to CRUD (Create, Edit, Update, Delete) this resource would be:

Action Urls Web MethodRestful Interface
Create /products POST products_url
Edit /products/1/edit POST edit_product_url
Update /products PUT product_url
Delete /products DELETE product_url

Then to get links to delete/edit a product through Restful interface, we can use link_to method as below:

link_to 'Edit', edit_product_path(product)

link_to 'Remove', product_path(product), :confirm => 'Are you sure?', 
        :method => :delete

By default, you can only use above interfaces in View layer. How can we use those in Controller layer?
  • In Rails 2, call through @template variable
    @template.link_to('Edit', edit_product_path(product))
  • In Rails 3, call through view_context method
    view_context.link_to('Delete', product_path(product), :confirm => 'Are you sure?', :method => :delete)
References

Tuesday, April 12, 2011

Code folding in Emacs

Code folding is a feature to wrap or unwrap a block of code. It can help you to see an overview of your code. This is a quick and simple way to have that feature in Emacs. I modified it a little bit to make it more flexible as the way I want.

;; code folding
(defun toggle-selective-display (level)
  (interactive "nEnter indentation level: ")
  (set-selective-display level)
  )

(global-set-key "\M-3" 'toggle-selective-display)
So, if I have this code:

class User < ActiveRecord
  has_many :projects
  has_many :assignments

  def full_name
    "#{first_name} #{last_name}"
  end

  def show_project_names
    projects.each do |prj|
      puts prj.name
    end
  end
  
end
then when I press Alt+3 4, all code that was indented at level 4 (4 spaces) will be wrapped as below:

class User < ActiveRecord
  has_many :projects
  has_many :assignments

  def full_name...
  end

  def show_project_names...
  end
  
end
if I press Alt+3 2, all code indented at level 2 will be wrapped:

class User < ActiveRecord...  
end
if I press Alt+3 0, wrapped code will be unwrapped again.

class User < ActiveRecord
  has_many :projects
  has_many :assignments

  def full_name
    "#{first_name} #{last_name}"
  end

  def show_project_names
    projects.each do |prj|
      puts prj.name
    end
  end
  
end
It's quite fun, right?

Sunday, March 20, 2011

A simple URL shortening algorithm

Shortening a URL is a convenient way to save long URL to make use of space when posting. It's especially popular on Twitter where message is limited to 140 words. Many websites provide this service such as tinyurl.com, bit.ly,...

There some gems or wrapper to use that service in your Rails app. But the shortened URLs belong to another domain (ex: http://bit.ly/ek8Hhe which belongs to bit.ly). If you want to make it belonged to your domain (ex: http://example.com/wfds7i), you must implement your own URLs shortener.

This is a simple way to do that.

Basically, the problem is:
given a URL, how to map it to a string which has pattern XXXXXX, where X belongs to {0..9a-zA-Z}. There would be 62^6 = 56800235584 such strings. That amount is almost enough.
Then the simple idea to solve that problem is:
map the URL to an integer in 1..62^6. That number must correspond to a string in space {XXXXXX} that could be calculated by using a 10-base to 62-base conversion algorithm (you can understand easily by figuring out how to convert a decimal number to hexa number). 

Here is an implementation of mine in Ruby:

class URLShortener
  CHARSET = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
  BASE = 62
  CODE_LENGTH = 6

  def self.encode(id)
    code = ""
    while (id > 0) do
      code = CHARSET[id % BASE].chr + code
      id = id / BASE
    end

    (code.length > CODE_LENGTH) ? "" : "0" * (CODE_LENGTH - code.length) + code 
  end

  def self.decode(code)
    return -1 if code.length != CODE_LENGTH
    id = 0
    for i in 0..(CODE_LENGTH-1) do
      n = CHARSET.index(code[i])
      return -1 if n.nil?
      id += n * (BASE ** (CODE_LENGTH - i - 1))
    end
    return id
  end
end

Making ajax form

Making an ajax form is a javascript code snippet that we see very frequently. So I've tried to write it as a pattern that could be easy to reuse when needed.

<form action="do_something" method="post" id="do_something_form"> 
  Field 1: <input type="text" name="field1" id="field1" /> 
  <br /> 
  Field 2: <input type="text" name="field2" id="field2"/> 
  <br /> 
  <input type="submit" value="Submit" /> 
</form> 

<script>
  jQuery('#do_something_form').submit(function(event){
    // stop normal form submitting
    event.preventDefault();
  
    // get form's fields
    var field1_val = jQuery('#field1').val(),
        field2_val = jQuery('#field2').val(),
        url        = jQuery(this).attr('action');

    // send request
    jQuery.post(url,                   // ajax url
                {
                  field1: field1_val,
                  field2: field2_val
                },                     // data passed to server side
                function(response){    // callback 
                  // do things with response data, ex:
                  alert(response["message"]);
                  if (response["success"] == true) {
                    // ... 
                  }
                },
                "json"                 // response's format 
    );
  });
</script>
 
Then at backend:
 
def create
  @form = params
  
  if some_condition
    @result = {
      :success    => true,
      :message    => "Resource has been created successfully!",
      :other_data => {}
    }
  else
    @result = {
      :success => false,
      :message => "Creation failed! Please try again!"
    }
  end

  render :text => @result.to_json
end
Reference:
http://api.jquery.com/jQuery.post/

Sunday, March 13, 2011

Tổng quan về kinh tế học #1




Nói đến kinh tế là nói đến quá trình sản xuất, trao đổi hàng hóa, của cải vật chất và tinh thần trong xã hội. Thông qua quá trình đó, con người tương tác với nhau để tạo ra của cải vật chất hay những giá trị mới, tái phân bổ chúng, phục vụ cho các nhu cầu trong cuộc sống, rồi lại lao động và lại tái tạo ra những giá trị mới khác,... Đó là tổng thể của cái gọi là nền kinh tế. Làm sao để nền kinh tế tạo ra thật nhiều giá trị? Làm sao để quá trình sản xuất và lưu thông hàng hóa hiệu quả hơn?... Đó là những vấn đề của kinh tế học.

Một trong những yếu tố cơ bản của nền kinh tế là thị trường và tiền tệ.

Nơi hàng hóa được đem ra trao đổi, mua bán được gọi là thị trường. Vd: thị trường bất động sản, thị trường chứng khoán, chợ,...

Hàng hóa có thể được trao đổi, mua bán trên thị trường dưới nhiều hình thức:
- Trao đổi trực tiếp:
   vd: 1 trâu đổi 100 thúng lúa
- Trao đổi gián tiếp (quy đổi sang một loại hàng hóa trung gian)
   vd: 1 trâu <-> 100 thúng lúa
         1 bò <-> 80 thúng lúa
         1 căn nhà <-> 1000 thúng lúa
   hoặc,
         1 trâu <-> 1 lượng vàng
         1 bò <-> 0.5 lượng vàng
         1 nhà <-> 100 lượng vàng

Yếu tố đóng vai trò trung gian trong trao đổi hàng hóa được gọi là tiền tệ. Thời xưa, tiền tệ cũng chính là hàng hóa (lúa, vàng, bạc,...), bản thân chúng có giá trị nội tại. Khi kinh tế ngày càng phát triển, nhiều hình thức tiền tệ khác ra đời làm cho quá trình trao đổi, lưu thông hàng hóa thuận tiện, nhanh chóng hơn như tiền đồng, tiền giấy, tiền điện tử,... Tiền tệ trở về với đúng bản chất của nó: làm trung gian trong trao đổi hàng hóa, bản thân nó không có hoặc có rất ít giá trị nội tại. Vai trò trung gian, đại diện cho giá trị thực đó của tiền tệ được đảm bảo bởi nhà nước và pháp luật.

Saturday, March 12, 2011

Be careful writing hooks to Rack layer (middle ware layer) in Rails

I've got this error from a Rails project:

ERROR NoMethodError: private method `split' called for 0:Fixnum


Following the link below gave me some clues:
http://rack.lighthouseapp.com/projects/22435/tickets/52-read-error-nomethoderror-private-method-split-called-for-0fixnum


1. something's wrong in Rack layer
2. someone wrote incorrect code that hooked to Rack layer 

Case #1: rarely happened
Case #2: may be 

I did some search and found this code in authentication module:
response.headers["Expires"] = 0

This must be the cause of the error. 

Though the correct code is just: 
response.headers["Expires"] = "0"

it still took me over a day to find out and fix that bug!!! 

Simple but not simple!


A case of "Broken Pipe" error in Rails

Have you ever seen "Broken PIPE" error in Rails?

It somehow like this:


*** Exception Errno::EPIPE in Passenger RequestHandler (Broken pipe) (process 22235):
    from /usr/lib/ruby/gems/1.8/gems/passenger-2.1.2/lib/phusion_passenger/rack/request_handler.rb:67:in `write'


This error seems appear when your program interacts improperly with external programs. In my case, it happened to me in a project that I used XMLRPC library to remotely communicate with some web services.

My implementation for that feature is as following:
- create a XMLRPC connection object
- use that XMLRPC object every time calling a webservice API

There some other guys had been headache with that error also:
http://gaveen.owain.org/2008/04/errnoepipe-broken-pipe-mysql-error-in.html
http://stackoverflow.com/questions/1082166/exception-errnoepipe-in-passenger-requesthandler-broken-pipe
http://stackoverflow.com/questions/4351624/ruby-on-rails-errnoepipe-broken-pipe

and the reasons they found come around Passenger or Mysql.

In my case, I've got the same thing even with upgrading Passenger or migrating to another web server.
After a lot of retries, I've found that if I call the API very often, "Broken pipe" doesn't appear, but if I stop doing anything for a while and then call the API, it does happen. That observation led me to a guess: XMLRPC connection object may be broken after a certain amount of time (timeout). Excellently, that's exactly right! I then changed my implementation so that every time calling a webservice API, I use a newly created XMLRPC connection object instead of using the same XMLRPC connection object for every API calls.

That solved my problem!

Tuesday, February 22, 2011

Django Overview


(will add more soon...)

Saturday, February 19, 2011

#1 Setup a basic Django project

, I've been getting more and more interested in Django. I think why shouldn't I write a series about it. So, let's start with the basics first: how to setup Django development environment and build a basic Django project.

1. Setup environments
  • Install Python
       Get Python at http://www.python.org
       To verify if it's already installed, at console:
   $ python
   Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39) 
   [GCC 4.4.5] on linux2
   Type "help", "copyright", "credits" or "license" for more information.
  • Install Django
       Download a Django release at http://www.djangoproject.com/download/
   $ tar xzvf Django.tar.gz
   $ cd Django
   $ sudo python setup.py install 

       To verfiy what's your current Django's version
   $ python
   >>> import django
   >>> django.get_version()
    
2. Create a basic Django project 
  • To initialize project 
    $ django-admin.py startproject mysite
          This will create a basic Django project in mysite directory that looks as below
    mysite/
       __init__.py
       manage.py
       settings.py
       urls.py
  • Now run a development webserver to see if your Django site works
    $ python manage.py runserver
    Validating models...
    0 errors found.

    Django version 1.0, using settings 'mysite.settings'
    Development server is running at http://127.0.0.1:8000/
    Quit the server with CONTROL-C.       
  • Goto http://127.0.0.1:8000 on your Web browser, you'll see a "Welcome to Django" page. It's works!
So, you have setup a very basic Django project. If you want to make more complex stuffs, you may need to use a database backend, create some models, build your site's templates, mapping urls to what will process & response the corressponding templates (Django call this "views"), add some caching mechanisms to boost performance,...

Friday, February 18, 2011

How caching works in Django

Last time, I had a chance joining to a Django project. Django is a web framework written by Python. Though I've invested sometime before to study Django and been amazing with it, I haven't really done anything on Django yet. So that's a good opportunity for me to increase my Django knowledge. One of my tasks at start was to study caching supports in Django. Compare to what I knew with Ruby on Rails, I found that caching in Django is very strong and especially well-organized. So, it would be useful to put some summarizations down here.  
  1. General rule

    given a URL, try finding that page in the cache
    if the page is in the cache:
        return the cached page

    else:
        generate the page
        save the generated page in the cache (for next time)
        return the generated page

    1. More specific



    2. Advantages and disadvantages of caching backends

      Advantages Disadvantages
      Memcached
      - the fastest, most efficient type of cache available to Django
      - all cached data is stored directly in memory, so there’s no overhead of database or filesystem usage
      - cached data can be shared over multiple machines, so it’s excellent for scaling
      - being used by Facebook, Wikipedia,...


      - cached data is stored in memory, so it will be lost if server crashes (but it’s not critical because cached data is just temporary)
      - need to have memcached daemon along the way
      In-database cached
      - cached data is persistent
      - could use multiple databases for caching


      - setup & manage multiple caching databases would be tough
      Filesystem cached
      - cached data is persistent
      - simple to setup


      - may not scale well
      Local-memory cached
      - simple
      - not nescessary to have runing an external cache server as memcached

      - no cross-process caching means it’s not particularly memory-efficient
      - probably not a good choice for production, just nice for development


    3. Reference
      http://docs.djangoproject.com/en/1.2/topics/cache/