Blog Archive

Tag Cloud


Moai Cloud Sandbox Extension: GridFSFile added

Details
Category: News
Published on Friday, 15 June 2012 17:28
Written by Rob LaRubbio

For Moai Cloud, we recently added new functionality to our sandbox and have exposed most of the gridfs file object and all of the gridfs chunk object. With these additions you can now read chunks of large gridfs files into your lua handler without worring about running out of memory.

You can get more information about MongoDB's GridFS from their documentation.

To access a gridfile from either your code or content file store you would call:

local gridfile = moai.code.gridfs_file("file.txt")
local gridfile = moai.content.gridfs_file("file.txt")

The methods we expose on gridfile are:

function gridfile:chunk(chunk_num)
function gridfile:chunk_size()
function gridfile:content_length()
function gridfile:exists()
function gridfile:md5()
function gridfile:num_chunks()
function gridfile:upload_date()
function gridfile:data()
function gridfile:field(name)
function gridfile:metadata()

You can access the chunk be loading it via it's index. Indexes start at 0 so to get the first chunk:

local chunk = gridfile:chunk(0)

You can then get either the data or the size (length) of the chunk with these methods:

function gridchunk:len
function gridchunk:data

You can use code similar to this to read, iterate and stream the file. We'll be adding some higher level functions that make the streaming easier.

function http_msg(code, status, content_length)
  local headers = {}
  headers['content-length'] = content_length

  local raw = {}
  for k, v in pairs(headers) do
    if( "table" == type(v) ) then
      for l, w in pairs(v) do
        table.insert(raw, string.format('%s: %s', k, w))
      end
    else
      table.insert(raw, string.format('%s: %s', k, v))
    end
  end  return string.format('HTTP/1.1 %s %s\r\n%s\r\n\r\n', code, status, table.concat(raw, '\r\n'))
endfunction main(web, req)
  local gridfile = moai.content.gridfs_file("file.txt")
  local num_chunks = gridfile:num_chunks()
  local content_length = gridfile:content_length()  local chunk = nil  web.conn:reply(req, http_msg(200, "Ok", content_length))   for c = 0, (num_chunks - 1) do
    chunk = gridfile:chunk(c)    if chunk then
      web.conn:reply(req, chunk:data())
      posix.sleep(1)
    end
  end  web:close()  returnend

Enjoy, and as always let us know what you think in the forums and tell us want you want from Moai Cloud.