Lua Garbage Collection

Discussion about using Moai SDK - post questions, bugs and issues here.

Moderators: seebs, franciscotufro

Lua Garbage Collection

Postby rhoster » Fri Jun 08, 2012 1:56 pm

I've only recently started programming in Lua and want to make sure that I'm properly dereferencing all my variables so that they get properly garbage collected. That being said, I think I'm doing a bit of overkill. Here is an example:

Code: Select all
  1. -- SomeGameObject.lua

  2. SomeGameObject = {}

  3.  

  4. function SomeGameObject:new()

  5.         local setmetatable = setmetatable

  6.         local SomeGameObject_mt = Class(SomeGameObject) -- returns the metatable for the Class

  7.         return setmetatable( { AVariable = 0, AnotherVaribale = {} }, SomeGameObject_mt)

  8. end

  9.  

  10. function SomeGameObject:remove()

  11.         self["AVariable"] = nil

  12.         self["AnotherVariable"] = nil

  13.         self = nil

  14. end

  15.  



Code: Select all
  1. -- AnotherGameObject.lua

  2. require 'SomeGameObject'

  3.  

  4. AnotherGameObject= {}

  5.  

  6. function AnotherGameObject:new()

  7.         local setmetatable = setmetatable

  8.         local AnotherGameObject_mt = Class(AnotherGameObject) -- returns the metatable for the Class

  9.         return setmetatable( { SomeGameObject = SomeGameObject:new() }, AnotherGameObject_mt)

  10. end

  11.  

  12. function AnotherGameObject:remove()

  13.         local SomeGameObject = self["SomeGameObject"]

  14.         if SomeGameObject then

  15.                 SomeGameObject:remove() --removes the object through the function

  16.                 self["SomeGameObject"] = nil -- do I need to do this?  will it be nil from the previous function?

  17.                 SomeGameObject = nil -- is there a point to dereferencing the local variable?

  18.         end

  19.         self = nil

  20. end

  21.  



The questions are located in the AnotherGameObject:remove() function. Since the SomeGameObject class already has a remove() function that sets itself to nil, do I need to set the value to nil for the instance once I return from the SomeGameObject:remove() function? Likewise, do I need to nil the local version of SomeGameObject that I've created to test for whether the object exists and speed code up with local access?

I know these are more Lua questions than MOAI, but I thought I'd post here since it's related. I'd like to be able to remove self["SomeGameObject"] = nil and SomeGameObject = nil, relying on the fact that the instance remove function will do the job of dereferencing everything.

If anyone knows the answers to these off hand, I'd appreciate it. Thanks,
User avatar
rhoster
 
Posts: 205
Joined: Sat May 26, 2012 3:59 pm
Location: Santa Barbara, CA

Re: Lua Garbage Collection

Postby Tauto » Sun Jun 10, 2012 1:48 pm

You don't need the remove() function at all. If a variable is no longer accessible, it will be automatically GC'd.
Tauto
 
Posts: 5
Joined: Wed Apr 25, 2012 3:15 pm

Re: Lua Garbage Collection

Postby ibisum » Mon Jun 11, 2012 12:35 am

I think the point rhoster is trying to make is how to do GC when you want to do it - and yes, assigning to nil is the cleanest way to tell the Lua VM that var is qualified for GC in the next sweep. You only have to assign to nil once, and the GC will consider it qualified.
;
--
ibisum@gmail.com
Got a MOAI snippet? Please consider adding it to http://moaisnippets.info
User avatar
ibisum
 
Posts: 1026
Joined: Mon Oct 17, 2011 1:11 am
Location: Vienna, Austria

Re: Lua Garbage Collection

Postby rhoster » Mon Jun 11, 2012 2:09 am

Thanks for the replies. From what I gather from your responses, the GC can be done from the top - meaning that I only have dereference a variable in the global environment and it will be removed (even if references exist further down the chain).

So in my example, assuming AnotherGameObject existed in my main.lua, I could simply do:

Code: Select all
  1. myAnotherGameObject = AnotherGameObject:new()

  2. myAnotherGameObject = nil



And that Lua would know to get rid of both myAnotherGameObject and the SomeGameObject that was contained within it (since the SomeGameObject is no longer accessible by nature of its containing table AnotherGameObject no longer being accessible).
User avatar
rhoster
 
Posts: 205
Joined: Sat May 26, 2012 3:59 pm
Location: Santa Barbara, CA

Re: Lua Garbage Collection

Postby rhoster » Mon Jun 11, 2012 2:31 am

Actually, after rereading your post ibisum, I think what you're saying is that I don't need to double dereference it (so I could drop the second and third nils, but keep the function to allow me to have control of when it happens):

Code: Select all
  1. function AnotherGameObject:remove()

  2.         local SomeGameObject = self["SomeGameObject"]

  3.         if SomeGameObject then

  4.                 SomeGameObject:remove() --removes the object through the function

  5.         end

  6.         self = nil

  7. end

User avatar
rhoster
 
Posts: 205
Joined: Sat May 26, 2012 3:59 pm
Location: Santa Barbara, CA

Re: Lua Garbage Collection

Postby ibisum » Mon Jun 11, 2012 2:37 am

Perhaps this will be of some use to you:

http://www.lua.org/manual/5.1/manual.html#2.10
;
--
ibisum@gmail.com
Got a MOAI snippet? Please consider adding it to http://moaisnippets.info
User avatar
ibisum
 
Posts: 1026
Joined: Mon Oct 17, 2011 1:11 am
Location: Vienna, Austria

Re: Lua Garbage Collection

Postby rhoster » Mon Jun 11, 2012 12:46 pm

Thanks for the links, I've read the GC section (2.5 for the 5.2 release http://www.lua.org/manual/5.2/manual.html#2.5) a bunch of times but it still isn't fully clicking. Although I realize that I'm travelling down the rabbit hole here and should get back to actually writing the game code before I get too deep.

I did find these articles which give a better explanation of what is going on:

http://lua-users.org/wiki/GarbageCollectionTutorial
http://lua-users.org/wiki/OptimisingGarbageCollection
http://lua-users.org/wiki/GarbageCollection
http://lua-users.org/wiki/GarbageCollec ... lTimeGames

I admittedly have taken garbage collection for granted in PHP for the last few years and haven't had to deal with performance issues related to it since college.
User avatar
rhoster
 
Posts: 205
Joined: Sat May 26, 2012 3:59 pm
Location: Santa Barbara, CA

Re: Lua Garbage Collection

Postby ibisum » Mon Jun 11, 2012 1:56 pm

Well, fortunately, when you need to start worrying about GC is when there's a lot of game already written. ;)
;
--
ibisum@gmail.com
Got a MOAI snippet? Please consider adding it to http://moaisnippets.info
User avatar
ibisum
 
Posts: 1026
Joined: Mon Oct 17, 2011 1:11 am
Location: Vienna, Austria

Re: Lua Garbage Collection

Postby rhoster » Mon Jun 11, 2012 2:27 pm

Exactly :)
User avatar
rhoster
 
Posts: 205
Joined: Sat May 26, 2012 3:59 pm
Location: Santa Barbara, CA

Re: Lua Garbage Collection

Postby rhoster » Wed Jun 27, 2012 3:43 am

Interesting tidbit I just read here http://www.coronalabs.com/blog/2011/07/ ... ables-lua/

Will the global/upvalue table also be set to nil if you set it’s referencing local to nil? No


This means that I believe I would need to have all of the nils - because dereferencing a table locally (while pointing the global object) will only remove the reference to the global object, but won't actually affect it. In fact, setting a local reference to nil is apparently the only thing you can do to a local table reference that doesn't affect the actual global table.

Still need to wrap my head around this a bit more.
User avatar
rhoster
 
Posts: 205
Joined: Sat May 26, 2012 3:59 pm
Location: Santa Barbara, CA

Re: Lua Garbage Collection

Postby sgeos » Wed Jun 27, 2012 6:14 am

If anything in scope has a reference to the variable it can not be garbage collected. I hope this example helps:
Code: Select all
  1. $ lua

  2. Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio

  3. >  function newMessage(pMessage)

  4. >>   local result = function() print(pMessage) end

  5. >>   return result

  6. >> end

  7. >  newMessage("Test")()

  8. Test

  9. >  a = newMessage("Hi")

  10. >  a()

  11. Hi

  12. >  a = newMessage("Bye")

  13. >  a()

  14. Bye

  15. >  a, b = nil, a

  16. >  a()

  17. stdin:1: attempt to call global 'a' (a nil value)

  18. stack traceback:

  19.         stdin:1: in main chunk

  20.         [C]: ?

  21. >  b()

  22. Bye

  23. >  


This is what is going on:
  • Line 4: Set the local variable result equal to an unnamed function. You could call result() to print a message in this scope.
  • Line 5: Return the function stored in result. The local variable result is now nil.
  • Line 7: Create and call an unnamed "Test" function. There is no reference to the function after it prints the message and returns, so the "Test" function is eligible for garbage collection.
  • Line 9: Create an unnamed "Hi" function and store it in a. The "Hi" function is referenced by a and will not be garbage collected.
  • Line 12: Create an unnamed "Bye" function and store it in a. The "Bye" function is now referenced by a so this new the function will not be garbage collected. The "Hi" function is no longer referenced so it is eligible for garbage collection.
  • Line 15: Set a to nil, and b to the value of a (the "Bye" function).
  • Lines 17 ~ 20: a is indeed nil.
  • Lines 21 ~ 22: The "Bye" function is now referenced by b so it is not eligible for garbage collection even though a was set to nil.
Does this make sense to you?
Code: Select all
  1. $ lua

  2. Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio

  3. >  a = {}

  4. >  a.a = a

  5. >  print(a.a.a.a.a.a.a.a.a)

  6. table: 0x10010ab80

  7. >  a, b = nil, a

  8. >  print(a.a.a.a.a.a.a.a.a)

  9. stdin:1: attempt to index global 'a' (a nil value)

  10. stack traceback:

  11.         stdin:1: in main chunk

  12.         [C]: ?

  13. >  print(b.a.a.a.a.a.a.a.a)

  14. table: 0x10010ab80

  15. >  print(b.a.a.a.a.a.a.a.b)

  16. nil

  17. >  print(b.a.a.a.b.a.a.a.b)

  18. stdin:1: attempt to index field 'b' (a nil value)

  19. stack traceback:

  20.         stdin:1: in main chunk

  21.         [C]: ?

  22. >  b = nil

  23. >  


Is the table declared on line 3 ever garbage collected? When and why?
Any original code posted by me is released via the CC0 Public Domain Dedication. It is in the public domain. Do whatever you want with it.
User avatar
sgeos
 
Posts: 241
Joined: Sat Apr 28, 2012 4:42 am
Location: Married in Japan.

Re: Lua Garbage Collection

Postby rhoster » Wed Jun 27, 2012 6:13 pm

Thanks! This is an amazing example; here is my line by line take on what is happening:

  • line 3 - allocates memory for an empty table and sets the global a to reference that empty table
  • line 4 - sets the string key 'a' of global a to the global a, which sets the key 'a' of the empty table to reference itself
  • line 5 - prints the memory address of the empty table because the global a points to the empty table which looks up the key 'a' which is a reference to itself. This lookup process repeats until the chain finally reaches the last key 'a' which references the empty table
  • line 7 - sets the global a to nil and the global b to reference the empty table
  • line 8 - tries to print the global a which is set to nil causing an nil index error
  • line 13 - prints the memory address of the empty table because the global b references the empty table. The same chain of index lookups for the key 'a' repeats until it reaches the final key 'a' which references the empty table
  • line 15 - tries to follow the chain of index lookups for the key 'a' like in line 13 until the final key 'b' which being set to nil triggers the index of the global b which does indeed exist.

This is where I had to stop and do a little homework. After checking the Lua 5.2 Reference Manual I found this important part to the index metamethod:

Code: Select all
  1. function gettable_event (table, key)

  2.        local h

  3.        if type(table) == "table" then

  4.          local v = rawget(table, key)

  5.          -- if key is present, return raw value

  6.          if v ~= nil then return v end

  7.          h = metatable(table).__index

  8.          if h == nil then return nil end

  9.        else

  10.          h = metatable(table).__index

  11.          if h == nil then

  12.            error(···)

  13.          end

  14.        end

  15.        if type(h) == "function" then

  16.          return (h(table, key))     -- call the handler

  17.        else return h[key]           -- or repeat operation on it

  18.        end

  19.      end



The import part is line 3 which checks if the type is a table. If so, then it will do a rawget to see if there is actually a value present and return that value or nil. This is the reason that we get a nil on the final key 'b', because we are trying to index a table and coming up empty which is returning a nil. Since the global b does exist as a reference to a table type, we are able to index a key to determine if it exists - hence the nil. Since nil is a value that can be printed, it successfully prints.

  • line 17 - tries to follow the chain until the first key 'b' is reached. At this point, because the key 'b' is set to nil (which we know from our previous print in line 15), so we can no longer continue on our chain to the next key 'a'. This causes a nil index error for trying to access the key 'b'.
  • line 22 - set the global b to nil. This means that the only reference that exists to the empty table is the key 'a' which is a reference to itself so it is garbage collected

You have successfully explained ephemeron tables through example instead of the abstract and technical definition in the Lua 5.2 Reference Manual. After reading through the manual, that was one of the concepts that was still a bit fuzzy.

Our empty table is an ephemeron table starting at line 7 because at that point it contains weak keys and strong values. This is due to the fact that the actual empty table is the strong value and it is still reachable through the global b. The key 'a' is however weak because it is simply a reference to itself - only having any context while it is still reachable through the global b.

Once the global b is set to nil on line 22, the only reference for the key 'a' comes through its value (the empty table), so the key/value pair is set for garbage collection. Since that is the only key/value pair in our table and nothing references the table it is garbage collected :) Again, thanks very much for this great example that teaches through direct experience!! That is how I process new concepts and make them stick. I'm sure others will find this example very useful.
User avatar
rhoster
 
Posts: 205
Joined: Sat May 26, 2012 3:59 pm
Location: Santa Barbara, CA

Re: Lua Garbage Collection

Postby sgeos » Thu Jun 28, 2012 7:07 am

Correct!

Another thing I forgot to point out:
Code: Select all
  1. function newMessage(pMessage)

  2.   local result = function() print(pMessage) end

  3.   return result

  4. end

This forms a closure, so the string literals passed into newMessage() will not be released until the returned function is because they are referenced by the function.

I think you are understanding how garbage collection works in Lua, but here is just one more example:
Code: Select all
  1. $ lua

  2. Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio

  3. >  a = {name="apple"}

  4. >  b = {name="banana"}

  5. >  c = {name="carrot"}

  6. >  a.a, b.a, c.a = a, a, a

  7. >  a.b, b.b, c.b = b, b, b

  8. >  a.c, b.c, c.c = c, c, c

  9. >  function display(t)

  10. >>   print(t.name, t.a.name, t.b.name, t.c.name)

  11. >> end

  12. >  display(a); a = nil

  13. apple   apple   banana  carrot

  14. >  display(b); b = nil

  15. banana  apple   banana  carrot

  16. >  display(c); c = nil

  17. carrot  apple   banana  carrot

  18. >  

When is each of the tables released?
Any original code posted by me is released via the CC0 Public Domain Dedication. It is in the public domain. Do whatever you want with it.
User avatar
sgeos
 
Posts: 241
Joined: Sat Apr 28, 2012 4:42 am
Location: Married in Japan.

Re: Lua Garbage Collection

Postby rhoster » Thu Jun 28, 2012 1:51 pm

In this case, all three tables are released on line 16 when the reference to the global c is set to nil (releasing the keys 'a' and 'b' since they are no longer reachable.

While we're talking string literals and garbage collection, I wanted to note that I recently changed my method for concatenating strings based on the way that Lua handles string literals and their allocation. Previously I would do something like this:

Code: Select all
  1. local a_variable = "to print with"

  2. local another_variable = "until reaching"

  3. printAString(a_variable, another_variable)

  4.  

  5. function printAString(a_variable, another_variable)

  6. return "Some string "..a_variable.." the rest of the string "..another_variable.." the end."



If my understanding is correct, this will actually create all of the following strings in the scope of the function printAString:

"Some string"
"Some string to print with"
"Some string to print with the rest of the string"
"Some string to print with the rest of the string until reaching"
"Some string to print with the rest of the string until reaching the end."

Now, this might seem trivial in this case because those strings are released as soon as the function ends, but they are still being needlessly allocated. To solve this problem, I moved to the following method for concatenating strings:

Code: Select all
  1. local a_variable = "to print with"

  2. local another_variable = "until reaching"

  3. printAString(a_variable, another_variable)

  4.  

  5. function printAString(a_variable, another_variable)

  6. local concat = table.concat

  7. return concat({"Some string ",a_variable," the rest of the string ",another_variable," the end."})



The code above returns the same value but doesn't allocate a new string for each concatentation.
User avatar
rhoster
 
Posts: 205
Joined: Sat May 26, 2012 3:59 pm
Location: Santa Barbara, CA

Re: Lua Garbage Collection

Postby sgeos » Fri Jun 29, 2012 1:24 pm

Line 16 is correct. As for the strings, that is an interesting find.
Any original code posted by me is released via the CC0 Public Domain Dedication. It is in the public domain. Do whatever you want with it.
User avatar
sgeos
 
Posts: 241
Joined: Sat Apr 28, 2012 4:42 am
Location: Married in Japan.

Re: Lua Garbage Collection

Postby rhoster » Fri Jun 29, 2012 2:14 pm

I got the string stuff from here:

http://lua-users.org/wiki/StringsTutorial - under Concatenation
http://www.lua.org/pil/11.6.html

http://www.lua.org/pil/3.4.html

Remember that strings in Lua are immutable values. The concatenation operator always creates a new string...


http://stackoverflow.com/questions/1405583/concatenation-of-strings-in-lua
User avatar
rhoster
 
Posts: 205
Joined: Sat May 26, 2012 3:59 pm
Location: Santa Barbara, CA


Return to Moai SDK

Who is online

Users browsing this forum: No registered users and 0 guests

x