Jump to content


Photo

ATTENTION ALL MODDERS - Texture sizes and memory errors


  • Please log in to reply
20 replies to this topic

#1 Chris Ollis

Chris Ollis

    Master Director

  • Pioneers
  • 1631 posts

Posted 01 June 2009 - 02:27 PM

Hi Guys,

We have a vague suspicion that the problem some people have been having with Memory Errors and Java heap space is down to a few of the modding packs having quite astoundingly massive textures.

When you make packs, can you pay close attention to your textures. A floor or wall tile doesn't usually need to be bigger than 256x256, however I've stumbled on a few that are over 2000 pixels wide. While a key model in your scene can often shine with high res textures, tiling backgrounds are just a waste of resources, especially when they are each taking about 16mb of ram.

Please also try to use the DDS format, it is specifically designed for game textures, compresses very well for minimum disk space and streams across the graphics card like it's barely there. Unlike jpg files which while small on disk are a nightmare to uncompress (slowing the processor) and much bigger in memory than you'd think.
Additionally, DDS files carry their own mipmaps (tiny versions of the texture to use when you are far away) jpgs do not, so the processor/card has to calculate those as well. Additionally additionally (!) when a non square or 2:1 texture is force mipmapped, it fudges it a bit around the edges, squashing the texture in to a square which can cause seams and tiling errors. So any benefit of using a massive high quality image in the first place is lost.
Square images may be weird to people used to a world of landscape and portrait, but keeping them 64x64, 128x128, 256x256, 512x512 etc really does help your computer.

Also if you are using the same texture on multiple objects in your mod, put just one copy of that texture in a shared folder in the root of the Props folder. i.e. Chris's Mod/Props/Textures NOT Chris's Mod/Props/CAR1/Textures and Mod/Props/CAR2/Textures and Mod/Props/CAR3/Textures. Then you can browse to that folder in the Modders Workshop and save even more disk and card space.

You all may want to go check your user created addon folders and do some quick resizing.
And the more adventurous save those jpegs as dds files, remembering to change the file types in Modders Workshop as you go (otherwise they won't load anymore)
And finally, those who have created Mods, may be lovely enough to reupload fixed versions of the addon.

Sketchup converting people should pay a lot of attention to this as Sketchup models are often made for single renders, so huge textures are fine for their original intentions. Once the Sketchup process has been done, go in to the the file folders and check what you've got and if need be open them up in Photoshop and fix them.

I would love to give you a cut out and keep guide of how big textures should be for every asset type, but there will always be an exception. Just try to make all textures (and meshes) as small as possible before you find them rubbish to look at, that's all I can advise

Right, I've rambled enough, if anyone wants to copy this on to the Wiki, feel free.

Chris
Chris Ollis
Caretaker of www.moddingstorm.co.uk, the one stop shop for all your extra addon needs
Plus Minor Web Celebrity, Viral Guru, Advertising storyboarder, Toy designer, BBC and Channel4 comedy writer/animator :D

#2 davidwww

davidwww

    Master Director

  • Pioneers
  • 1881 posts

Posted 01 June 2009 - 02:35 PM

Oh bugger that's probably my modding packs, quite a few of them I didn't check the size of the textures, I just went OOH! there's a pretty one I'll just use that, since I have realized this I am trying to keep the texture size down (and the models too, that's why my WW2 pack is taking so long) and I don't know if I will ever have the time but I would like to retro fit my older packs some time?
Chris can you just go to a folder and swap the 2000 pixel texture for a smaller one and have it show up in Moviestorm or do you have to go through the workshop.

Regards
David

Moviestorm still Number One

#3 Overman

Overman

    Master Director

  • Pioneers
  • 562 posts

Posted 01 June 2009 - 02:39 PM

Excellent advice, Chris, thank you.

This issue is one that (I'm assuming) we should probably bear in mind when attaching a custom image to an object that accepts one. Paintings, etc. Am I right that a user texture will be loaded in similar fashion to one which is directly part of the materials of a prop? So while it might not be practical for the average user to prep their user images as DDS files, certainly getting the dimensions under control beforehand will help performance for the affected movie. (Yes?)
Phil "Overman" Rice
Zarathustra Studios - http://z-studios.com
You can connect with us On Twitter, On Facebook, On YouTube, and On Vimeo.

#4 Chris Ollis

Chris Ollis

    Master Director

  • Pioneers
  • 1631 posts

Posted 01 June 2009 - 02:41 PM

QUOTE (davidwww @ Jun 1 2009, 02:35 PM) <{POST_SNAPBACK}>
Chris can you just go to a folder and swap the 2000 pixel texture for a smaller one and have it show up in Moviestorm or do you have to go through the workshop.

Yes, that'll work as long as you don't change the file format (if its a jpeg, leave it as one)

I forgot to add to the post, Sketchup models are often made for single renders, so huge textures are fine for their original intentions. We might try and get a warning in to the Sketchup wizard in the future if things do seem huge.
Chris Ollis
Caretaker of www.moddingstorm.co.uk, the one stop shop for all your extra addon needs
Plus Minor Web Celebrity, Viral Guru, Advertising storyboarder, Toy designer, BBC and Channel4 comedy writer/animator :D

#5 Fulkster

Fulkster

    Master Director

  • Pioneers
  • 2292 posts

Posted 01 June 2009 - 02:48 PM

...our Sketchup Imports use png files (even when I use jpgs to build them)... by what you say, I'll assume the dds files are better, period?

Thanks for the help, Mr Ollis!!!
The next big thing: MOVIESTORM!

See my most recent movie: "Athena Project: Chapter 2 (Down the Rabbit-hole)"
http://www.moviestor...p;vid_id=102329

(Soon, "Athena Project 3: The Laws Of Robotics"

Moddingstorm! Cool Mods!

#6 Dave

Dave

    Critically acclaimed

  • Mods
  • 218 posts

Posted 01 June 2009 - 02:56 PM

I think a future version of the Modder's Workshop should warn about large textures?
Chief Bit Shoveller
Short Fuze Ltd

#7 reptor7

reptor7

    Master Director

  • Pioneers
  • 1299 posts

Posted 01 June 2009 - 03:07 PM

Thanks Chris. I'll be sure to check my stuff out. I know all my stuff been jpg so I'll begin converting to dds.

#8 davidwww

davidwww

    Master Director

  • Pioneers
  • 1881 posts

Posted 01 June 2009 - 03:45 PM

Can the Sketchup importer be made to convert any non .dds textures, this would help any users not very technical who just import the .skp file without modification

Regards
David

Moviestorm still Number One

#9 Chris Ollis

Chris Ollis

    Master Director

  • Pioneers
  • 1631 posts

Posted 01 June 2009 - 04:03 PM

QUOTE (davidwww @ Jun 1 2009, 03:45 PM) <{POST_SNAPBACK}>
Can the Sketchup importer be made to convert any non .dds textures, this would help any users not very technical who just import the .skp file without modification

Regards
David

At the moment no, the dds format is a tricky one for us to get rights to manipulate. You'll need to go to nVidia's site here http://developer.nvi...ds_plugins.html to get the photoshop tools which simply provides you with a File/Save As *.dds instead of *.jpg (as well as Normal Map filters)

Or find the similar one for Gimp if you don't have Adobe's expensive toy
Chris Ollis
Caretaker of www.moddingstorm.co.uk, the one stop shop for all your extra addon needs
Plus Minor Web Celebrity, Viral Guru, Advertising storyboarder, Toy designer, BBC and Channel4 comedy writer/animator :D

#10 Ben_S

Ben_S

    Master Director

  • Mods
  • 4811 posts

Posted 01 June 2009 - 04:24 PM

QUOTE (davidwww @ Jun 1 2009, 03:45 PM) <{POST_SNAPBACK}>
Can the Sketchup importer be made to convert any non .dds textures, this would help any users not very technical who just import the .skp file without modification

Regards
David


I put in a feature request, but it might not be something that we have time to work on anytime soon, especially if it is 'tricky' as Chris says.
Ben Sanders
Moviestorm Ltd

#11 tree

tree

    Master Director

  • Pioneers
  • 924 posts

Posted 01 June 2009 - 05:31 PM

QUOTE (Chris Ollis @ Jun 2 2009, 12:27 AM) <{POST_SNAPBACK}>
Hi Guys,

We have a vague suspicion that the problem some people have been having with Memory Errors and Java heap space is down to a few of the modding packs having quite astoundingly massive textures.

When you make packs, can you pay close attention to your textures. A floor or wall tile doesn't usually need to be bigger than 256x256, however I've stumbled on a few that are over 2000 pixels wide. While a key model in your scene can often shine with high res textures, tiling backgrounds are just a waste of resources, especially when they are each taking about 16mb of ram.

Please also try to use the DDS format, it is specifically designed for game textures, compresses very well for minimum disk space and streams across the graphics card like it's barely there. Unlike jpg files which while small on disk are a nightmare to uncompress (slowing the processor) and much bigger in memory than you'd think.
Additionally, DDS files carry their own mipmaps (tiny versions of the texture to use when you are far away) jpgs do not, so the processor/card has to calculate those as well. Additionally additionally (!) when a non square or 2:1 texture is force mipmapped, it fudges it a bit around the edges, squashing the texture in to a square which can cause seams and tiling errors. So any benefit of using a massive high quality image in the first place is lost.
Square images may be weird to people used to a world of landscape and portrait, but keeping them 64x64, 128x128, 256x256, 512x512 etc really does help your computer.

Also if you are using the same texture on multiple objects in your mod, put just one copy of that texture in a shared folder in the root of the Props folder. i.e. Chris's Mod/Props/Textures NOT Chris's Mod/Props/CAR1/Textures and Mod/Props/CAR2/Textures and Mod/Props/CAR3/Textures. Then you can browse to that folder in the Modders Workshop and save even more disk and card space.

You all may want to go check your user created addon folders and do some quick resizing.
And the more adventurous save those jpegs as dds files, remembering to change the file types in Modders Workshop as you go (otherwise they won't load anymore)
And finally, those who have created Mods, may be lovely enough to reupload fixed versions of the addon.

Sketchup converting people should pay a lot of attention to this as Sketchup models are often made for single renders, so huge textures are fine for their original intentions. Once the Sketchup process has been done, go in to the the file folders and check what you've got and if need be open them up in Photoshop and fix them.

I would love to give you a cut out and keep guide of how big textures should be for every asset type, but there will always be an exception. Just try to make all textures (and meshes) as small as possible before you find them rubbish to look at, that's all I can advise

Right, I've rambled enough, if anyone wants to copy this on to the Wiki, feel free.

Chris


All important stuff.
one bad texture or having 1 high-poly prop can spoil a good set.

feature request ;-):
1.a props error checker (checking for oversized textures and polysize for example) so we can weed out the trouble-makers :-) I think there is a start on this with ctrl+G....
So a new user can somehow block the troublesome props from loading to help their positive experience.

2. wire frame view on selected props... to increase the power :-)
3. more moviestorm animation/props artists on staff at short-fuss :-)

........

About Props:
Using a "Grouped Textures holder" in an addon (addon/3dtreebsg/Textures) will be cool if i have textures i need to re-use on a few props.
I find myself making odd props then transferring them over to a new addon later.
Though if i move one prop to a new addon I'll probably have to manually transfer it over to the new addon then reset the material directories.(or maybe it just finds the root of the present addon... i'll have to explore that...)

About Texture sizes:
In my latest addon...(3dtreeBSG) All the diffuse textures were 1024size ...The Dif textures started to look ugly at 512 so i used 1024-only having one texture (material[0] ) per prop. I've also built the Normal/Emissive/Specular at the same size. Would you recommend a reduced and differing normal/specular/etc. sizes for these textures?





#12 daleh

daleh

    Master Director

  • Pioneers
  • 810 posts

Posted 01 June 2009 - 05:48 PM

Thanks for the info Chris,

can you offer any advice on the various settings for DDS? I have been using 'DDS converter 2' to convert textures between DDS and Jpeg/Png formats for editing, now I have the Gimp Plugin, but there seems many options when you come to save (or convert) a texture back to DDS, such as various compression type, DXT1, DXT3, DTX5, ATI1 etc etc, along with Mipmap filter and Colour Correction?

#13 Chris Ollis

Chris Ollis

    Master Director

  • Pioneers
  • 1631 posts

Posted 01 June 2009 - 07:23 PM

QUOTE (daleh @ Jun 1 2009, 05:48 PM) <{POST_SNAPBACK}>
Thanks for the info Chris,

can you offer any advice on the various settings for DDS? I have been using 'DDS converter 2' to convert textures between DDS and Jpeg/Png formats for editing, now I have the Gimp Plugin, but there seems many options when you come to save (or convert) a texture back to DDS, such as various compression type, DXT1, DXT3, DTX5, ATI1 etc etc, along with Mipmap filter and Colour Correction?


If there is no Alpha (transparency) then set it to DXT1 if there IS transparency set it to DXT5 smile.gif


And Tree, yes, 1024 is bigger than we use for trees, especially if you are using alpha/transparency. Halving your normal and specular is also a good idea, although at that res you can probably get away with a quarter size specular
Chris Ollis
Caretaker of www.moddingstorm.co.uk, the one stop shop for all your extra addon needs
Plus Minor Web Celebrity, Viral Guru, Advertising storyboarder, Toy designer, BBC and Channel4 comedy writer/animator :D

#14 Reacher

Reacher

    Master Director

  • Pioneers
  • 307 posts

Posted 01 June 2009 - 07:25 PM

QUOTE (daleh @ Jun 1 2009, 11:48 AM) <{POST_SNAPBACK}>
Thanks for the info Chris,

can you offer any advice on the various settings for DDS? I have been using 'DDS converter 2' to convert textures between DDS and Jpeg/Png formats for editing, now I have the Gimp Plugin, but there seems many options when you come to save (or convert) a texture back to DDS, such as various compression type, DXT1, DXT3, DTX5, ATI1 etc etc, along with Mipmap filter and Colour Correction?


Daleh,
From the Moviestorm Wiki on Textures:

On DDS texture formats, Ben_S wrote:
DXT5 with alpha (for things with transparencies)
DXT1 without alpha (for those without transparencies)

DXT1 with alpha will not work on a significant number of graphics cards, so don’t use it.

I also updated the article with the reasons to prefer dds over the other supported formats (Creating a texture | Moviestorm Wiki).

-- Reacher

#15 lucindamc123

lucindamc123

    Master Director

  • Pioneers
  • 9177 posts

Posted 01 June 2009 - 07:48 PM

QUOTE (Reacher @ Jun 1 2009, 07:25 PM) <{POST_SNAPBACK}>
Daleh,
From the Moviestorm Wiki on Textures:

On DDS texture formats, Ben_S wrote:
DXT5 with alpha (for things with transparencies)
DXT1 without alpha (for those without transparencies)

DXT1 with alpha will not work on a significant number of graphics cards, so don’t use it.

I also updated the article with the reasons to prefer dds over the other supported formats (Creating a texture | Moviestorm Wiki).

-- Reacher


I have Adobe Photoshop and I am going to convert all my textures to DDS files sized 256 X 256 for those props I have in this movie specifically. I think I will start with just making a new Sketchup model of all the props I am using in Urban Dream.

then I will impor them as new props to a new addon and see if that works. I will let you know. I do have hopes this will work. Also even though if I use a Sketchup 3d Warehouse model, I replace all the photo type testures with Sketchup textures, I do not remove them from the model for the list of textures in model. So the importer may be trying to find those textures too.

I do rebuild all of the buildings now that i get from 3d Warehouse, building windows and details instead of just using a photograph of a building as I have found the buildings with photograph details do not work well at all. The Cafe Adler I did, I made from scratch though so that one should work fine. I will also check all of the models that I am using from David for the Berlin Wall and see if that is the problem with textures too large. thanks for the help.

I believe I may have found part of the problem. As you said David, you didn't pay attention to the size of the jpgs you were using as images and the ones for the berlin Wall are huge -- 2300 X 2300 or larger in some cases so that may be the problem along with unused textures I still had loaded with my own buildings.

#16 Norrie

Norrie

    Novice director

  • Members
  • 70 posts

Posted 01 June 2009 - 07:57 PM

I'm frankly amazed that there are so many old Lionhead stalwarts making these errors!
I thought DDS and power of two were requisites for any modding in any programme; game, machinima, or graphics of any kind.

Still, we all know modding is something to learn, not something to complain about not being able to do. Like it's someone else's fault.



#17 daleh

daleh

    Master Director

  • Pioneers
  • 810 posts

Posted 02 June 2009 - 08:23 AM

QUOTE (Norrie @ Jun 1 2009, 07:57 PM) <{POST_SNAPBACK}>
Still, we all know modding is something to learn, not something to complain about not being able to do. Like it's someone else's fault.


Hmm, I think you may have posted this reply in the wrong thread wink.gif , I do not see any complaining here. What I do see is a developer offering some good advice, which may have solved the issues of several other threads involving slowdown and loading issues of MS, and the community embracing it and asking for additional info.

Out of which we have had some good suggestions for adding texture size checking to the Sketchup importer and a feature request for built in DDS conversion.

All good, positive stuff.


#18 bongoman

bongoman

    Master Director

  • Pioneers
  • 767 posts

Posted 12 June 2009 - 12:00 PM

DISABLE UNUSED MODS
Is a practice I've recently started - I've made a lot of mods by now, and haven't cleared out all the extra ones yet, so I turn them off in the loadup screen in MS and save all that wasted RAM.


#19 Poulet Noir

Poulet Noir

    Master Director

  • Pioneers
  • 1136 posts

Posted 12 June 2009 - 12:36 PM

QUOTE (bongoman @ Jun 12 2009, 12:00 PM) <{POST_SNAPBACK}>
DISABLE UNUSED MODS
Is a practice I've recently started - I've made a lot of mods by now, and haven't cleared out all the extra ones yet, so I turn them off in the loadup screen in MS and save all that wasted RAM.

Amen to that!

How tricky would it be to give us the ability to activate / deactivate add-on packs from within Moviestorm? I imagine that would add to the RAM overhead, but could save resources in the long run.

I'm pretty much expecting the answer "that's near-impossible in Moviestorm's current form", but (1) you never know..., and (2) it's something to consider for the future wink.gif

PN

"Come back Black Chicken! Leave us an egg or something..." Love at First Bite
 

Bat avatar used by kind permission of Jon Huckeby. Check out his blog!


#20 ed

ed

    Master Director

  • Pioneers
  • 373 posts

Posted 04 May 2010 - 09:48 AM

QUOTE (Poulet Noir @ Jun 12 2009, 12:36 PM) <{POST_SNAPBACK}>
Amen to that!

How tricky would it be to give us the ability to activate / deactivate add-on packs from within Moviestorm? I imagine that would add to the RAM overhead, but could save resources in the long run.

I'm pretty much expecting the answer "that's near-impossible in Moviestorm's current form", but (1) you never know..., and (2) it's something to consider for the future wink.gif

PN


A button in Moviestorm's Set View that could identify all the currently used/unused addons would be handy, especially it it also offered a "disable all unused AddOns?" switch :-)

(Could that go on the list of feature requests?)


  • Please log in to reply


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users