Encoding Settings Help

[shaqsalazar]

Hi, i'm asking help from all top encoders

 

PC SPECS

 

CPU: i7-4770k OC @ 4.2 ghz (liquid cool)

PSU: 850W

RAM: 16 GB (4x4 GB) @ 1600

HDD: 120 GB SSD, 8 TB HDD (multiple drives)

GPU: NVidia Geforce GTX 660ti

 

Here are my h264/x264 encoding settings, i want to know if i'm using good settings. If not can you tell me what i'm doing wrong, and explain why these setting or settings need be change and what will changing help improve?

 

Encoding Mode = automated 2 pass TESTING

Profile = High

Level = 4.1

Preset = Very Slow

Tune = Animation CHANGING

Ref. Frame = 3

B-Frames = 12

B-Frame Bias = 0

Adaptive B-Frames = Optimal

B-Frames Pyramid = None

Adaptive I-Frames Decisions = Yes(checked)

Scene Change Sensitivity = 100

P-Frames Weighted Prediction = K-Means ( weightp = 3)

Keyframes Interval Control = Automatic

Deblocking = Yes

Deblocking Strength = 1 CHANGING

Deblocking Threshold = -2 CHANGING

All Slicing settings = 0

Entropy Coding = CABAC

Weighted Prediction for B-Frames = Yes (checked)

Bitrate Variance = 0.8

Quantizer Compression = 0.6

Temp. Blur of Est. Frame Complexity = 20

Temp. Blur of Quantizer after CC = 0.5

Rate Control Lookahead = 250

Fade Compensation = 0.5

Macroblock Tree Rate Control = Yes (checked)

QP = 0:45:4

AQ Mode = Variance AQ

AQ Strength = 0.6 RE-TESTING

M.E Range = 64

M.E Algorithm = UMH

Subme = 10

Direct MV Predicition Mode = Automatic

Trellis Quantization = Always

Psy-RD Strength = 0.4 RE-TESTING

Psy-Trellis Strength = 0

Macroblocks Partitions = ALL

Mixed Reference Frames = Yes (checked)

DCT Decimation = Yes (checked) REMOVED

Fast P-Skip Detection = yes (checked)

Psychovisual Optimzations = yes (checked)

color primaries = BT709 REMOVED

transfer = BT709 REMOVED

color matrix = BT709

 

x264 Encoder Threads = 8

lookahead threads = 12

Enable Slice-based Threading = Yes (checked)

Enable CPU Optimizations = Yes (checked)

 

Encoding TIME:

1-Pass = 22.xx fps,  20-30 mins

2-Pass = 10-16  fps 40 mins - 1 hr

 

I ask because i feel like i reach the limit of how much i can improve my encodes. I want to know if i truly reach the limit or if there is a lot more i can improve on.

 

its for 1080p encodes

 

Encoder Settings Explained http://mewiki.project357.com/wiki/X264_Settings

 

Thank you

 

Edited July 19, 2014 by shaqsalazar

[ChoeyXD]

Have you ever thing about two-pass encode?

[StaticX]

I think CRF might help, but as you know it's unpredictable in filesizes

[shaqsalazar]

I think CRF might help, but as you know it's unpredictable in filesizes

well from what i read from h264 doc's its not as efficient as automated 2-pass when it comes to compress and quality

Have you ever thing about two-pass encode?

what do you mean? i am encoding in 2-pass

 

or 

 

are seeing it's bad?

[undertehker]

 

 

well from what i read from h264 doc's its not as efficient as automated 2-pass when it comes to compress and quality

 

Not sure about this, but what koby said makes sense to me.

 

 

Because bitrate encoding is constant bitrate. CRF gives bitrate where bitrate is needed; so one scene may have 2000 while another scene only has 500, leaving overall bitrate much lower and saving space. Bitrate encoding either starves the video or bloats it (because, say you use 800 bitrate; one scene may have needed 1500 so that scene got starved while another scene only needed 200 so that scene got bloat). While CRF if used properly only gives the required bitrate to each scene.

[Baal]

This:

Deblocking Strength = 1

Deblocking Threshold = -2

 

why are you assuming that all anime you encode need less deblocked blocks but with higher strength?

 

Deblocking 0:0 should not be touched unless your source needs it.

 

----

Fade Compensation = 0.5

So, which custom x264 build are you using?

Edited July 18, 2014 by Baal

[JohnFlower]

2pass? Lol~ I thought everyone in the anime scene had moved on from that.

Edit: Clarified my position.

Edited July 18, 2014 by JohnFlower

[Koby]

2pass? Lol~ I thought everyone had moved on from that, especially in the anime scene.

The only people I know who still use 2-pass is Shaq and DarkDream, but DarkDream also encodes to a specifically set filesize too.

Neither of them have upgraded to ASS subs either. Shaq uses PGS and DarkDream uses SRT.

[shaqsalazar]

 

2pass? Lol~ I thought everyone had moved on from that, especially in the anime scene.

The only people I know who still use 2-pass is Shaq and DarkDream, but DarkDream also encodes to a specifically set filesize too.

Neither of them have upgraded to ASS subs either. Shaq uses PGS and DarkDream uses SRT.

 

i do use ass in most releases its usually dual-subs with untouched BD pgs and ass/ssa subs

 

Ok thanks for the feedback ill start experimenting with CRF again. Before the start of this year most of my encodes where encoded at CRF @ 16

Edited July 19, 2014 by shaqsalazar

[shaqsalazar]

This:

Deblocking Strength = 1

Deblocking Threshold = -2

 

why are you assuming that all anime you encode need less deblocked blocks but with higher strength?

 

Deblocking 0:0 should not be touched unless your source needs it.

 

----

Fade Compensation = 0.5

So, which custom x264 build are you using?

http://astrataro.wordpress.com/category/encode/x264/x264-bin/ here's what i use. I use it because it shows/unlocks more settings in x264, and is more optimize. 

 

Ok thanks for the tip you are right

 

So any other things i need to change?

Edited July 19, 2014 by shaqsalazar

[Baal]

ME Range 64

 

You don't need it higher than 24, above 24 is meant for use with ESA/TESA algorithms (or maybe 32 for the ease of mind but with 64 you are wasting time)

 

color primaries = BT709

transfer = BT709

color matrix = BT709

 

You can drop transfer/color primaries, important is color matrix.

 

AQ Strength = 0.6

Psy-RD Strength = 0.4

 

These two need most attention and require test encodes to find good settings for each series.

Edited July 19, 2014 by Baal

[Koby]

Just a couple things I decided to note for now that I saw:

Tune = Animation

If you're going to use a specific tune rather than just alter the stuff yourself, I find that "Tune = Grain" works much better at preserving details, which is something you'd want to do.

B-Frames = 12

I may not be correct in this, but I was under the impression that less than that is recommended. I suggest 8 b-frames personally.

Ref. Frame = 3

This is rather low. I'd suggest somewhere between 4-6 ref. I tend to just go with 5.

Lookahead = 250

While this may be max setting and offers greater results; the extra time it takes vs. the benefits just isn't worth it. Use something like 60 for better benefits between speed and quality.

DCT Decimation = Yes (checked)

If you're using Trellis (which you are), then you should turn off DCT Decimation.

[Baal]

Ref. Frame = 3

This is rather low. I'd suggest somewhere between 4-6 ref. I tend to just go with 5.

 

If he wants to stay within level 4.1 then 4 is max.

[Koby]

Ref. Frame = 3

This is rather low. I'd suggest somewhere between 4-6 ref. I tend to just go with 5.

 

If he wants to stay within level 4.1 then 4 is max.

That doesn't seem to be the case from what I've read. Apparently some Blu-ray discs are encoded at 6 ref frames @ level 4.1 as well.

http://doom10.org/index.php?topic=69.0

Perhaps maybe you are confusing the fact that some standalone players can't play more than 4 reference frames and can't work with video above 4.1 and making the wrong conclusion that 4.1 can't have more than 4 reference frames.

Though they seem to be saying in that thread for compatibility sakes you might as well stick with 9 for 720p and 4 for 1080p so just for the sake of compatibility with various devices you'd probably want to stick with ref=4.

It's 4am, I need sleep.

[JohnFlower]

Nobody sticks with standards unless they're encoding specifically for hardware players (which is stupid, imo).

But meh. This is just another one of those 'which is best' threads. There's no real point unless you start providing video files of source vs encode and start using crf for anime and provide the actual line you are encoding with, not a bunch of meaningless settings that you may or may not have set yourself.

[Baal]

 

 

Ref. Frame = 3

This is rather low. I'd suggest somewhere between 4-6 ref. I tend to just go with 5.

 

If he wants to stay within level 4.1 then 4 is max.

 

That doesn't seem to be the case from what I've read. Apparently some Blu-ray discs are encoded at 6 ref frames @ level 4.1 as well.

http://doom10.org/index.php?topic=69.0

Perhaps maybe you are confusing the fact that some standalone players can't play more than 4 reference frames and can't work with video above 4.1 and making the wrong conclusion that 4.1 can't have more than 4 reference frames.

Though they seem to be saying in that thread for compatibility sakes you might as well stick with 9 for 720p and 4 for 1080p so just for the sake of compatibility with various devices you'd probably want to stick with ref=4.

It's 4am, I need sleep.

 

 

 

I'm going by this:

http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels

 

My point is that if Shaq aims for compatibility with hardware players then his best bet is to stay within level constraints.

[BluerGost]

@Shaq

Its good that you are trying to improve your encode. Which will hopefully help your encode to be less bloated . xD

[shaqsalazar]

ME Range 64

 

You don't need it higher than 24, above 24 is meant for use with ESA/TESA algorithms (or maybe 32 for the ease of mind but with 64 you are wasting time)

 

color primaries = BT709

transfer = BT709

color matrix = BT709

 

You can drop transfer/color primaries, important is color matrix.

 

AQ Strength = 0.6

Psy-RD Strength = 0.4

 

These two need most attention and require test encodes to find good settings for each series.

 

So after me range 32 it becauses unnecessary to encode any higher than, ok thanks

 

 

Just a couple things I decided to note for now that I saw:

Tune = Animation

If you're going to use a specific tune rather than just alter the stuff yourself, I find that "Tune = Grain" works much better at preserving details, which is something you'd want to do.

B-Frames = 12

I may not be correct in this, but I was under the impression that less than that is recommended. I suggest 8 b-frames personally.

Ref. Frame = 3

This is rather low. I'd suggest somewhere between 4-6 ref. I tend to just go with 5.

Lookahead = 250

While this may be max setting and offers greater results; the extra time it takes vs. the benefits just isn't worth it. Use something like 60 for better benefits between speed and quality.

DCT Decimation = Yes (checked)

If you're using Trellis (which you are), then you should turn off DCT Decimation.

 

ok ill try grain

 

as for b-frames isnt it meant for compression? so isnt high values good?

 

as for ref i always put it at 4, but when the encoding is done, for so reason the encoder sets it ref 3 (be it mod x264 or normal)

 

as for lookahead i have the CPU power so it does not bother me. With all the setting set before it takes 45 mins - 1 hr to encode one episodes. With the settings you guys gave me, that time should decrease. But i'm not worry about encoding time atm.

 

ok i will turn off DCT

 

Thanks a lot

 

 

 

Ref. Frame = 3

This is rather low. I'd suggest somewhere between 4-6 ref. I tend to just go with 5.

 

If he wants to stay within level 4.1 then 4 is max.

 

That doesn't seem to be the case from what I've read. Apparently some Blu-ray discs are encoded at 6 ref frames @ level 4.1 as well.

http://doom10.org/index.php?topic=69.0

Perhaps maybe you are confusing the fact that some standalone players can't play more than 4 reference frames and can't work with video above 4.1 and making the wrong conclusion that 4.1 can't have more than 4 reference frames.

Though they seem to be saying in that thread for compatibility sakes you might as well stick with 9 for 720p and 4 for 1080p so just for the sake of compatibility with various devices you'd probably want to stick with ref=4.

It's 4am, I need sleep.

 

 

that is the standard of ref, but like i stated above the encoder sets auto back to ref 3 so regardless if i choose ref 4 will set it back to ref 3.

 

 

 

 

 

 

 

 

 

I'm going by this:

http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels

 

My point is that if Shaq aims for compatibility with hardware players then his best bet is to stay within level constraints.

 

 

and yes Baal i am aiming for compatibility i already try these settings on my Blu-ray home theater system and games consoles, they all play smoothly with any lag or buffer problems

 

 

Thanks a lot guys i'm going to try your settings and compare it with my old settings will post screen shots, between the new settings and BD source and my old settings and BD source

[professa X]

Respect...just...respect. So gotta get get a pc like yours if I wanna do bd encodes.