Jump to content


Photo

Perfromance issues with larger sets


  • Please log in to reply
19 replies to this topic

#1 daleh

daleh

    Master Director

  • Pioneers
  • 810 posts

Posted 11 November 2009 - 01:39 PM

Since 1.1.7 the improvement in performance has been great, but I have mostly been working on small 'indoor' sets.

Anyway, i needed a scene where someone is being followed, and so created a brand new street set, kinda in an capital 'I' shape, so theres a T-Juntion at both ends of the main street allowing me to film in any direction on the main street.

Now its quite a detailed set, with road markings, a wall surrounding a car park, trafic lights and buildings used to hide the horizon, but I didn't think i had gone 'too' mad with it.

Anyway, I then started creating my following scene, with just 2 actors, nothing too complex, just them walking along one slightly behind the other with a few pauses to look and then cross a road.

But right from the start it felt very sluggish when adding anything, such as actor walks or camera cuts, and if modifing anything, like dragging an actors walk.

By the time the scene was 2 mins long, maybe not even quite 2 mins, new camera cuts were taking 10 seconds or more to appear. The scene itself plays perfectly smoothly, theres no 'graphics' slowdown at all, its just horrid delays during the creation process of the scene.

I have tried it with all shaders turned off and even lowered the graphics setting but it really makes no difference to the delays i am encountering, and my PC is reasonably decent, E6750 CPU, 2Gb ram, Nvidia GT8800gts, XP sp3.

Just wondered if anyone else had noticed any issues when working with larger sets or if maybe something i have done in the set is causing the issue?



#2 Walvince

Walvince

    Master Director

  • Pioneers
  • 311 posts

Posted 11 November 2009 - 02:39 PM

Oh! That is EXACTLY the problem I have with a set too.
Each action takes 30 seconds (place an actor, change the walk style, ...). I thought it was a problem with my set but finally no?
This is sooo long to build the scene, it's been 1 week I am working on it. The slowest thing is the delay before the loading time, because it does not appear immediately. The soft freeze for about 15-20s after doing something and only then there is the loading logo.
Sorry in advance for my bad english^^


#3 Fulkster

Fulkster

    Master Director

  • Pioneers
  • 2292 posts

Posted 11 November 2009 - 03:07 PM

I recently (in the past month or so, with 1.1.7.1) did a little vid (The Robbery... just a little telling of a joke someone told me) using a set similar (I think) to the one you describe. I didn't have any problems... (using three cameras).

(My first thought is that maybe you have a prop(s) which has a large picture for a texture?)

I went back in and tried changing long and short paths of characters walking... it seems that there is a 5-7 second delay as you attempt to drag the character while the path gets divided into two (but you can't manipulate the character)... but after the path (in the timeline) is divided you can do you manipulating. (But that's not really anything to do with your camera loads.)

Our PCs are pretty similar; I have 3 gig of RAM, so that might be some of the difference... you could maybe be coming up against memory issues?

How many cameras are you using? Maybe you could use more jumps with fewer cameras?
The next big thing: MOVIESTORM!

See my most recent movie: "Athena Project: Chapter 2 (Down the Rabbit-hole)"
http://www.moviestor...p;vid_id=102329

(Soon, "Athena Project 3: The Laws Of Robotics"

Moddingstorm! Cool Mods!

#4 lucindamc123

lucindamc123

    Master Director

  • Pioneers
  • 9177 posts

Posted 11 November 2009 - 03:08 PM

When I made I Would Do Anything for Love, I created a set with over 20 houses, most of them modded and I have about 15 to 20 people on the set along with moving cars. I had no problems with that set. Before I cleaned up my hard drive and reinstalled Moviestorm, I had a lot of problems. Now I have no problems with speed or anything. In fact I can even be rendering a movie in the background in a program that uses 80 percent of my memory and still work on a Moviestorm movie. I can even move back and forth from the starmaker to the movie, redress puppets, etc, add new puppets and the program runs really fast now.

I have a geforce 9400 video card, 3 gigs of ram and 2 terabyte drives. That might make the difference I don't know but I had all that before I reformated my C drive and reinstalled all my software. This was not something I did by choice. I had a horrible virus this summer and it was the only thing I could do. I have about 20 k worth of software on my computer and I had to reinstall all of it. I never want to have to do that again. Actually all my software now runs better.

It might be something in your computer or on a drive that is using your memory.


http://www.moviestor...p;vid_id=102854

#5 daleh

daleh

    Master Director

  • Pioneers
  • 810 posts

Posted 11 November 2009 - 04:52 PM

QUOTE (Fulkster @ Nov 11 2009, 03:07 PM) <{POST_SNAPBACK}>
I recently (in the past month or so, with 1.1.7.1) did a little vid (The Robbery... just a little telling of a joke someone told me) using a set similar (I think) to the one you describe. I didn't have any problems... (using three cameras).

(My first thought is that maybe you have a prop(s) which has a large picture for a texture?)

I went back in and tried changing long and short paths of characters walking... it seems that there is a 5-7 second delay as you attempt to drag the character while the path gets divided into two (but you can't manipulate the character)... but after the path (in the timeline) is divided you can do you manipulating. (But that's not really anything to do with your camera loads.)

Our PCs are pretty similar; I have 3 gig of RAM, so that might be some of the difference... you could maybe be coming up against memory issues?

How many cameras are you using? Maybe you could use more jumps with fewer cameras?


Only using 1 Camera and 2 Actors in the scene, and I am not using any custom props on this set, only Short Fuse stuff.

I have used an image on a poster on a bus stop, I wonder if thats it, i'll try removing it and see, or re-sizing the picture in Gimp.

#6 julian_gold

julian_gold

    Critically acclaimed

  • Staff
  • 204 posts

Posted 11 November 2009 - 04:53 PM

If you guys could zip us up and send us a scene, we could run some tests to see where it's dragging?
If I have seen further it is because I have stood on the shoulders of the giant pile of corpses of the myopic masses.

#7 daleh

daleh

    Master Director

  • Pioneers
  • 810 posts

Posted 11 November 2009 - 04:57 PM

QUOTE (lucindamc123 @ Nov 11 2009, 03:08 PM) <{POST_SNAPBACK}>
When I made I Would Do Anything for Love, I created a set with over 20 houses, most of them modded and I have about 15 to 20 people on the set along with moving cars. I had no problems with that set. Before I cleaned up my hard drive and reinstalled Moviestorm, I had a lot of problems. Now I have no problems with speed or anything. In fact I can even be rendering a movie in the background in a program that uses 80 percent of my memory and still work on a Moviestorm movie. I can even move back and forth from the starmaker to the movie, redress puppets, etc, add new puppets and the program runs really fast now.

I have a geforce 9400 video card, 3 gigs of ram and 2 terabyte drives. That might make the difference I don't know but I had all that before I reformated my C drive and reinstalled all my software. This was not something I did by choice. I had a horrible virus this summer and it was the only thing I could do. I have about 20 k worth of software on my computer and I had to reinstall all of it. I never want to have to do that again. Actually all my software now runs better.

It might be something in your computer or on a drive that is using your memory.


http://www.moviestor...p;vid_id=102854



It could be that my PC is due for its 6 month re-install that I always do, but i am not having any issues in other programs. In fact I just purchased Dragon Age: Origins, and in terms of graphics it is ofc light years ahead of Moviestorm, with very detailed textures and huge open areas full of detail, and my PC runs it smooth as silk with no slow down at all. Though as I already stated MS also runs very smooth in terms of graphics on my machine, when you are just panning around or letting the scene play in Camer View, its only when adding new stuff or changing exisitng camer cuts or actor actions that i am getting delays and slugishness.



#8 lucindamc123

lucindamc123

    Master Director

  • Pioneers
  • 9177 posts

Posted 11 November 2009 - 05:37 PM

Did you try the memory trick that one of the staff here posted that allows Moviestorm to use more of your memory? I did that. Movietorm is very memory efficient. I can run other programs that are using sometimes 90% of my memory and run Moviestorm along side it with no problems.

#9 daleh

daleh

    Master Director

  • Pioneers
  • 810 posts

Posted 12 November 2009 - 10:31 AM

QUOTE (julian_gold @ Nov 11 2009, 04:53 PM) <{POST_SNAPBACK}>
If you guys could zip us up and send us a scene, we could run some tests to see where it's dragging?


What exactly do you need, just the folder containing the movie or the custom stock set folder as well..?

And where do I send them to?

I have since redone this scene, splitting it into 3 scenes, this has made working on each scene segment a little faster, but even when split up things are still sluggish when adding or modifying actions/camera cuts.

I still have the original 'all in one' scene though that i can send you once i know exactly what to send and where.

QUOTE (lucindamc123)
Did you try the memory trick that one of the staff here posted that allows Moviestorm to use more of your memory? I did that. Movietorm is very memory efficient. I can run other programs that are using sometimes 90% of my memory and run Moviestorm along side it with no problems


Yeah Lucinda I have been using the 1.1.7 memory fix, I have 2G of ram and have it set to 1024, just as a test I put it back to the default of 512, but it made no difference, didn't make the scene respond any better or worse.






#10 lucindamc123

lucindamc123

    Master Director

  • Pioneers
  • 9177 posts

Posted 12 November 2009 - 01:28 PM

QUOTE (daleh @ Nov 12 2009, 10:31 AM) <{POST_SNAPBACK}>
What exactly do you need, just the folder containing the movie or the custom stock set folder as well..?

And where do I send them to?

I have since redone this scene, splitting it into 3 scenes, this has made working on each scene segment a little faster, but even when split up things are still sluggish when adding or modifying actions/camera cuts.

I still have the original 'all in one' scene though that i can send you once i know exactly what to send and where.



Yeah Lucinda I have been using the 1.1.7 memory fix, I have 2G of ram and have it set to 1024, just as a test I put it back to the default of 512, but it made no difference, didn't make the scene respond any better or worse.


That is really strange. I do run XP. Do you have XP on your computer? I don't know why mine works so well now. But it all happened after I reformated my hard drive.


#11 daleh

daleh

    Master Director

  • Pioneers
  • 810 posts

Posted 12 November 2009 - 01:50 PM

QUOTE (lucindamc123 @ Nov 12 2009, 01:28 PM) <{POST_SNAPBACK}>
That is really strange. I do run XP. Do you have XP on your computer? I don't know why mine works so well now. But it all happened after I reformated my hard drive.


Its XP I am using, for the most part MS runs fine, and always did, but especially so after 1.1.7, its just this particular set casuing me issues, though at the end of the day the scene plays fine, its just the creation process which is slow due to it taking several seconds for any changes I make or things I add to appear.

Anyway I have now emailed the scene and set to Julian, so will see what he makes of it.

After you mentioned about having 20 building in a scene you had used with no issues I counted mine, there are just over 30 buildings form the Town Buildings Addon, each between 2 and 3 storeys high plus roofs, plus some shops and various street dressing, including road markings for all 3 streets in the set. The set also takes up the entire MS world, in fact some of the buildings extend just beyond the gridspace to help mask/hide the horizon.

Its possible I have just built a set a bit beyond what MS an handle, if thats true then i dont mind suffering the delays, in fact I have already split the scene up into 3 seperate scenes so that the slowdown is reduced on each one. i dont mind working with those restraints to get the look I want, but if its some kind of bug or corruption thats happened to my set then obviously it would be nice to have that fixed or know what it is that I did to cause the problem... smile.gif

#12 Walvince

Walvince

    Master Director

  • Pioneers
  • 311 posts

Posted 12 November 2009 - 04:08 PM

I don't know if it can help to understand the problem but yesterday i tried to delete prop after prop to see if one were causing problems. So I deleted at least 20 buildings and 10 cars and finally the loading time is passed from 25s to 10s.
So I do not have much things on the set now but there is still a 10s delay for each action, strange (i don't have this problem for other big sets)
Sorry in advance for my bad english^^


#13 julian_gold

julian_gold

    Critically acclaimed

  • Staff
  • 204 posts

Posted 12 November 2009 - 04:37 PM

As you may know, it's my job to keep tabs on Moviestorm performance. My tests this week have been on what we call "retakes" - this is the process by which we turn the commands that you give our objects into actual chains of animations, and it is quite a time-consuming process. For example, consider a walk - it consists of many different snippets of animations stitched together in as seamless a fashion as possible. When you move a target point, the walk gets recalculated. In fact during a retake everything is recalculated, and this may seem like overkill; if you only change something near the end, why do you have to process the entire scene? Well the answer is that determining which bits of the scene are (un)affected by which operations can be a more time-consuming process than recalculating the lot anyway! It can also be more error-prone, and more complex (hence bug-prone too). So for now, when we retake (and we retake a lot), we do the entire scene, from time = 0 to the end of the scene.

In order to mitigate the cost of the retake, we've put some effort into making sure we are reducing the amount of recalculation done. This is done "locally" to each activity (rather than globally as above). One of the areas this is harder to optimise is when the activity depends on some resource - which could be eg a texture or a data file lurking on your hard drive. In order to be robust, we have to put checks in to see if these resources have changed - so if you change an animation (directly or indirectly), that change propagates into your scene. Large (ie long) scenes result in lots of resources, and hence Moviestorm spends a lot of time in the retake gathering info about resources. At the moment, for reasons I won't bore you with the details of, this is more costly than we'd like. In essence there are multiple file systems in place, and they all have their costs. We intend to fix the inefficiencies as soon as we can; but they are actually quite insidious because we do a lot of file-related activity and so the changes need to be made in lots of places. But the good news is that we are on the case, and with v1.2 due out soon, we'll have time to take stock of these issues, hopefully to come in a subsequent release asap.
If I have seen further it is because I have stood on the shoulders of the giant pile of corpses of the myopic masses.

#14 daleh

daleh

    Master Director

  • Pioneers
  • 810 posts

Posted 12 November 2009 - 06:58 PM

Thanks for looking at my scene and set Julian and giving such a fast response.

Based on what you are saying then the longer a scene is, combined with how much you have altered things, redone camera cuts, moved things on the timeline, adjusted walk waypoints etc, affects the overall speed of the scene as you contine to work on it..?

I assume then that Set complexity and size also has an effect?, as with the set I sent you its more sluggish right from the start, ie, creating a new scene in a new movie, and placing the very first actor down and then adjusting the actors start position, immeadietly on that set its more sluggish to respond than say on a small living room set. Obviously it then continues to get worse and worse as the set grows in size, but i have had small sets with lots of actors on them doing lots more complex stuff and not had anyway near the slowdown.

With all this in mind is there anything we as users can do to reduce this problem in the short term, such as the way we create scenes or design sets, and what about about hardware, would increasing my ram help (I currently have 2Gb on Windows XP sp3), and what about defragging hard drives, is that kind of stuff likely to help improve things..?

#15 Walvince

Walvince

    Master Director

  • Pioneers
  • 311 posts

Posted 12 November 2009 - 07:02 PM

And i would add does multi-core helps for the speed?
Sorry in advance for my bad english^^


#16 lucindamc123

lucindamc123

    Master Director

  • Pioneers
  • 9177 posts

Posted 13 November 2009 - 03:25 AM

QUOTE (daleh @ Nov 12 2009, 01:50 PM) <{POST_SNAPBACK}>
Its XP I am using, for the most part MS runs fine, and always did, but especially so after 1.1.7, its just this particular set casuing me issues, though at the end of the day the scene plays fine, its just the creation process which is slow due to it taking several seconds for any changes I make or things I add to appear.

Anyway I have now emailed the scene and set to Julian, so will see what he makes of it.

After you mentioned about having 20 building in a scene you had used with no issues I counted mine, there are just over 30 buildings form the Town Buildings Addon, each between 2 and 3 storeys high plus roofs, plus some shops and various street dressing, including road markings for all 3 streets in the set. The set also takes up the entire MS world, in fact some of the buildings extend just beyond the gridspace to help mask/hide the horizon.

Its possible I have just built a set a bit beyond what MS an handle, if thats true then i dont mind suffering the delays, in fact I have already split the scene up into 3 seperate scenes so that the slowdown is reduced on each one. i dont mind working with those restraints to get the look I want, but if its some kind of bug or corruption thats happened to my set then obviously it would be nice to have that fixed or know what it is that I did to cause the problem... smile.gif



I don't think it is too much for Moviestorm. The scene I made had all large homes and were all mods so that means they were higher poly than the Moviestorm buildings and it ran fine.

The only thing you can really do is to have a new movie for each scene and make smaller scenes. You might use the same set for each scene but you want to make different movies because the more animation you add to a scene makes the program work slower.

Things like nightclub scenes where you want to have puppets dancing slows things down a bit too.

If it is really impossible to have a lot of animated puppets on a large set, do chromakey. I use chromakey a lot. I didn't with that scene in I Would Do Anything for Love because I didn't have to.

#17 julian_gold

julian_gold

    Critically acclaimed

  • Staff
  • 204 posts

Posted 13 November 2009 - 09:41 AM

QUOTE (Walvince @ Nov 12 2009, 07:02 PM) <{POST_SNAPBACK}>
And i would add does multi-core helps for the speed?


We have no explicit use of multi-core, but Java can be a clever old beast and leverage it without even asking. However, the most likely time it will do this is in multithreaded code, and although Moviestorm is multi-threaded, it is not done so in such a fashion that would particularly benefit retake performance. Since the issue at the moment is one of resource tracking, performance will be beholden to things like

  • Your OS (Mac variant/Vista/XP/32/64)
  • The relationship between Java and the native file system
  • Hard drive speed, capacity, cache size, level of fragmentation
  • Background services competing for access (eg disk indexing)


and not CPU characteristics per se. Our efforts will be to minimise the amount of file (and consequently string) operations we perform, and this should start to speed things up.
If I have seen further it is because I have stood on the shoulders of the giant pile of corpses of the myopic masses.

#18 julian_gold

julian_gold

    Critically acclaimed

  • Staff
  • 204 posts

Posted 13 November 2009 - 09:54 AM

On the CPU side of things, a large set made from lots of small objects will be slow - not only because there are lots of things to draw, but because for shadows and the like we compute the extents of the world every frame, and that can be quite expensive. So rather than built big sets, construct several small partial ones that hide their nature by use of camera angles (just like making a real movie!) This will not only speed things up a bit, but your shadows will look better smile.gif

Just to be clear: it's not always the length of time the scene lasts that hurts, it's also the amount of activity that matters. Two people staring at each other doing nothing for 10 minutes is really no more expensive than for 1 minute. On the other hand, a longer walk requires necessarily more computation, because we are performing route-planning algorithms and stitching many small animations together heuristically. Using improvisation can be expensive too - this gets recalculated during retakes, and the more gaps you have between activities, the more gestures you get (which in turn can bring in new animation resources). So one tip is to get your major action done with improv OFF, and then switch it on and tweak it towards the end. Short controlled bursts!
If I have seen further it is because I have stood on the shoulders of the giant pile of corpses of the myopic masses.

#19 daleh

daleh

    Master Director

  • Pioneers
  • 810 posts

Posted 13 November 2009 - 10:14 AM

Thanks for the feedback Julian, knowing all this stuff helps a lot.

The set I built is large simply becuase I wanted the freedom to film in one street from any angle without seeing the horizon, hence the T-Juntion at either end, and the follow scene was then built around the set. The slowdowns have been a bit frustrating, but now that i have split the scene in 3 smaller chunks its much better, and still worth the frustration of the slowdown to have that camera angle freedom.

I think in future if doing this kind of thing I will build the rough set layout only, then do the scene, then add the details such as road markings and signs etc.

I am also thinking of doing a mod to have street markings as textures instead of using the prop version, as this may also speed up the set creation process, lining up all those double yellow lines was another story entirely.. wink.gif - though they look good in the end...smile.gif

Time to bite the bullit and start defragging my drive.. sad.gif

#20 lucindamc123

lucindamc123

    Master Director

  • Pioneers
  • 9177 posts

Posted 13 November 2009 - 03:57 PM

QUOTE (daleh @ Nov 13 2009, 10:14 AM) <{POST_SNAPBACK}>
Thanks for the feedback Julian, knowing all this stuff helps a lot.

The set I built is large simply becuase I wanted the freedom to film in one street from any angle without seeing the horizon, hence the T-Juntion at either end, and the follow scene was then built around the set. The slowdowns have been a bit frustrating, but now that i have split the scene in 3 smaller chunks its much better, and still worth the frustration of the slowdown to have that camera angle freedom.

I think in future if doing this kind of thing I will build the rough set layout only, then do the scene, then add the details such as road markings and signs etc.

I am also thinking of doing a mod to have street markings as textures instead of using the prop version, as this may also speed up the set creation process, lining up all those double yellow lines was another story entirely.. wink.gif - though they look good in the end...smile.gif

Time to bite the bullit and start defragging my drive.. sad.gif



Yes decorating the set after you have all the animation and voice overs in will help out a lot.


  • Please log in to reply


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users