This article from stackoverflow suggests linked lists (which I think are intended to be used with pointers) to append allocatable arrays on the fly with more or less elements.
http://stackoverflow.com/questions/8384406/how-to-increase-array-size-on-the-fly-in-fortran
That's all fine but ehh. Those of us coming from Matlab background might not understand what's going on with pointers, linked lists, etc. nor do we understand how Matlab actually dynamically allocates arrays. Since I don't care about the former (right now), the latter question can be answered easily. An array is allocated according to the number of elements contained within. If you want to add another element on the bottom, Matlab will allocate a new spot in memory with 1 more element, copy the old array to that physical memory location, then deallocate the original array. This is poor programming and very inefficient but you'd never notice for small arrays. For large arrays, your code will crawl.
C, fortran, most other languages of interest (I can't remember how python does it) force you to declare the exact number of elements you intend to store in an array before you can store any elements at all. In the tutorial guides, you'll see whole sections about allocating and deallocating variables and think that it's all unnecessary - because matlab just works! Except even really good matlab can be very slow. I recently converted a code to Fortran and got a 99.4% increase in speed over a well written, matlab code.
So if you're in C or Fortran, what can we do if we need to allocate more space on the fly? One option is (written in Fortran)
allocate(temp(size(myVariable)))
temp = myVariable
deallocate(myVariable)
allocate(myVariable(size(temp)+1))
myVariable(1:temp) = temp
This is doing exactly what Matlab does, but you have to code it yourself. If this is in a loop, and you're doing this every iteration, things will be slowed down considerably but RAM conservation will be maximized.
Why not allocate for a large fixed size ahead of time? allocate(myVariable(100)). Because if you only have 10 elements to store, then 90 percent of your memory usage is waste.
Here's my solution
allocate with a number
allocate(myVariable(10))
counter = 10
then
pcounter = 0
do i=1,Total,1
myVariable(i) = i
pcounter = pcounter + 1
if pcounter = counter then
allocate(temp(size(myVariable)))
temp = myVariable
deallocate(myVariable)
allocate(myVariable(size(temp)+10))
myVariable(1:temp) = temp
counter = counter + 10
end if
end do
this gives a buffer of 10 so you don't reallocate every iteration, but at worst, only every 10 iterations
the opposite can be done to deallocate on the fly
if counter - 10 > pcounter then
allocate(temp(size(myVariable)))
temp = myVariable
deallocate(myVariable)
allocate(myVariable(size(temp)-10))
myVariable(1:temp) = temp
counter = counter - 10
end if
this approach isn't too computationally intensive and it saves a ton of RAM for large simulations. Best of all, it's pretty straightforward in any language. I've even used it in Matlab when reading experimental data from arduino boards.
Tuesday, March 3, 2015
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment