Discussion:
[RFC] 10 bits
(too old to reply)
Maksym Veremeyenko
2017-01-29 12:00:15 UTC
Permalink
Raw Message
Hi,

i am currently thinking about capturing and playback 10-bit yuv, but
have no idea how start handling of it.

ffmpeg has no format for packed 16-bit Y/Cr/Cb samples - only planar.
v210 is also good, but not used as registered in MLT.

have you any ideas?

--
Maksym Veremeyenko
Brian Matherly
2017-01-29 20:03:35 UTC
Permalink
Raw Message
> Hi,
>
> i am currently thinking about capturing and playback 10-bit yuv, but
> have no idea how start handling of it.
>
> ffmpeg has no format for packed 16-bit Y/Cr/Cb samples - only planar.
> v210 is also good, but not used as registered in MLT.
>
> have you any ideas?
>
> --
> Maksym Veremeyenko

Using our current pixel formats, your best option is to convert to RGB immediately since it has 24bpp - and thus would lose the least data.


Alternately, we could add a new pixel format. I've been thinking about this some lately. I think we could add one pixel format that maps to AV_PIX_FMT_YUV422P16BE. Any bit depth greater than 8 could be converted to this format for processing in MLT. That would at least allow easy conversions to/from avformat. Personally, I don't care for interleaved formats for processing. I think interleaved is fine for transmission (e.g. SDI), but I feel that planar is easier to deal with when processing images in software. That's just my opinion.

~Brian
Dan Dennedy
2017-01-29 21:18:50 UTC
Permalink
Raw Message
I am fine with adding an image format to MLT, but I want it to be something
very useful and comprehensive so we do not soon need yet another format.
So, with that said, why not YUV 4:4:4 instead of 4:2:2? How including alpha
such as YUVA444P16 or RGBA64?
For more efficient conversion with the popular v210 format,
https://wiki.multimedia.cx/index.php/V210 and other sources say v210 is
little endian. So, choosing the 'LE' variant might more sense.

On Sun, Jan 29, 2017 at 12:10 PM Brian Matherly <***@brianmatherly.com>
wrote:

> > Hi,
> >
> > i am currently thinking about capturing and playback 10-bit yuv, but
> > have no idea how start handling of it.
> >
> > ffmpeg has no format for packed 16-bit Y/Cr/Cb samples - only planar.
> > v210 is also good, but not used as registered in MLT.
> >
> > have you any ideas?
> >
> > --
> > Maksym Veremeyenko
>
> Using our current pixel formats, your best option is to convert to RGB
> immediately since it has 24bpp - and thus would lose the least data.
>
>
> Alternately, we could add a new pixel format. I've been thinking about
> this some lately. I think we could add one pixel format that maps to
> AV_PIX_FMT_YUV422P16BE. Any bit depth greater than 8 could be converted to
> this format for processing in MLT. That would at least allow easy
> conversions to/from avformat. Personally, I don't care for interleaved
> formats for processing. I think interleaved is fine for transmission (e.g.
> SDI), but I feel that planar is easier to deal with when processing images
> in software. That's just my opinion.
>
> ~Brian
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Mlt-devel mailing list
> Mlt-***@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mlt-devel
>
Brian Matherly
2017-01-29 21:35:28 UTC
Permalink
Raw Message
YUVA444P16 or RGBA64 would cover all the bases.
LE vs BE just depends on whether you are more likely to operate on the samples as uint16_t or as uint8_t. uint16_t would be more convenient with LE because the two bytes would be in the correct byte order, uint8_t would be more convenient with BE because the MSB would come first (on x86 platform).

I would be OK with either.
FWIW, I predict that we will want to implement slice parallel conversions when dealing with these larger formats since even simple memory copies can take several milliseconds when you are dealing with large buffers. The global thread pool would be timely for this.
~Brian

From: Dan Dennedy <***@dennedy.org>
To: Brian Matherly <***@brianmatherly.com>; Maksym Veremeyenko <***@m1stereo.tv>; mlt-devel <mlt-***@lists.sourceforge.net>
Sent: Sunday, January 29, 2017 3:18 PM
Subject: Re: [Mlt-devel] [RFC] 10 bits

I am fine with adding an image format to MLT, but I want it to be something very useful and comprehensive so we do not soon need yet another format. So, with that said, why not YUV 4:4:4 instead of 4:2:2? How including alpha such as YUVA444P16 or RGBA64?For more efficient conversion with the popular v210 format, https://wiki.multimedia.cx/index.php/V210 and other sources say v210 is little endian. So, choosing the 'LE' variant might more sense.
On Sun, Jan 29, 2017 at 12:10 PM Brian Matherly <***@brianmatherly.com> wrote:

> Hi,
>
> i am currently thinking about capturing and playback 10-bit yuv, but
> have no idea how start handling of it.
>
> ffmpeg has no format for packed 16-bit Y/Cr/Cb samples - only planar.
> v210 is also good, but not used as registered in MLT.
>
> have you any ideas?
>
> --
> Maksym Veremeyenko

Using our current pixel formats, your best option is to convert to RGB immediately since it has 24bpp - and thus would lose the least data.


Alternately, we could add a new pixel format. I've been thinking about this some lately. I think we could add one pixel format that maps to AV_PIX_FMT_YUV422P16BE. Any bit depth greater than 8 could be converted to this format for processing in MLT. That would at least allow easy conversions to/from avformat. Personally, I don't care for interleaved formats for processing. I think interleaved is fine for transmission (e.g. SDI), but I feel that planar is easier to deal with when processing images in software. That's just my opinion.

~Brian
Maksym Veremeyenko
2017-02-11 15:18:28 UTC
Permalink
Raw Message
On 29.01.2017 23:35, Brian Matherly wrote:
> YUVA444P16 or RGBA64 would cover all the bases.
yes, in a case of capturing native SDI's 10-bit it would require
additional chroma interpolation twice: first time after capturing, next
time before encoding.

--
Maksym Veremeyenko
Maksym Veremeyenko
2017-02-11 15:20:16 UTC
Permalink
Raw Message
any objection against introducing *mlt_image_yuv422p10* ?

--
Maksym Veremeyenko
Dan Dennedy
2017-02-11 17:33:09 UTC
Permalink
Raw Message
On Sat, Feb 11, 2017 at 7:20 AM Maksym Veremeyenko <***@m1stereo.tv>
wrote:

> any objection against introducing *mlt_image_yuv422p10* ?
>
> no
Brian Matherly
2017-02-11 22:23:56 UTC
Permalink
Raw Message
On 2/11/2017 9:20 AM, Maksym Veremeyenko wrote:
> any objection against introducing *mlt_image_yuv422p10* ?
>

I am in full support of 422 and planar formats. Just wondering about the
bit packing.

What would be the corresponding AV format?

Would it be 16 bpp with 6 bits unused? BE or LE?

If it is going to be 16bit samples, then why not call it
mlt_image_yuv422p16 and use it for any bit depth between 9 and 16?

~Brian
Falkenberg Thomas
2017-02-11 23:00:43 UTC
Permalink
Raw Message
origin might come from the 10bit sdi word...
See my old SDI consumer...

Thomas



-------- Ursprüngliche Nachricht --------
Von: Brian Matherly <***@brianmatherly.com>
Datum: 11.02.17 23:24 (GMT+01:00)
An: Maksym Veremeyenko <***@m1stereo.tv>, mlt-***@lists.sourceforge.net
Betreff: Re: [Mlt-devel] [RFC] 10 bits

On 2/11/2017 9:20 AM, Maksym Veremeyenko wrote:
> any objection against introducing *mlt_image_yuv422p10* ?
>

I am in full support of 422 and planar formats. Just wondering about the
bit packing.

What would be the corresponding AV format?

Would it be 16 bpp with 6 bits unused? BE or LE?

If it is going to be 16bit samples, then why not call it
mlt_image_yuv422p16 and use it for any bit depth between 9 and 16?

~Brian



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Mlt-devel mailing list
Mlt-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mlt-devel

________________________________

This communication, issued by Broadcasting Center Europe S.A. (BCE), is confidential and we do not waive confidentiality by mistransmission. If you have received it by mistake please notify the sender immediately by reply and then delete it permanently from your system without copying it, using it for any purposes or disclosing its contents to any other person. Thank you for your cooperation. Any views expressed in this message are those of the individual sender and may not necessarily reflect the views of BCE. Emails are not secure and cannot be guaranteed to be error free as they can be intercepted, amended, lost or destroyed, or contain viruses. Anyone who communicates with us by email is taken to accept these risks.
Maksym Veremeyenko
2017-02-12 08:38:55 UTC
Permalink
Raw Message
On 12.02.2017 0:23, Brian Matherly wrote:
> On 2/11/2017 9:20 AM, Maksym Veremeyenko wrote:
> > any objection against introducing *mlt_image_yuv422p10* ?
> >
>
> I am in full support of 422 and planar formats. Just wondering about the
> bit packing.
>
> What would be the corresponding AV format?
>
> Would it be 16 bpp with 6 bits unused? BE or LE?
>
AV_PIX_FMT_YUV422P10LE

> If it is going to be 16bit samples, then why not call it
> mlt_image_yuv422p16 and use it for any bit depth between 9 and 16?

introducing *mlt_image_yuv422p16* that equal to AV_PIX_FMT_YUV422P16LE
(http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavutil/pixdesc.c;h=3b9c45d035f5f2399a652a00b41832f55a8febe1;hb=HEAD#l1496)
fine to me...

--
Maksym Veremeyenko
Brian Matherly
2017-02-12 13:04:09 UTC
Permalink
Raw Message
On 2/12/2017 2:38 AM, Maksym Veremeyenko wrote:
> On 12.02.2017 0:23, Brian Matherly wrote:
>> On 2/11/2017 9:20 AM, Maksym Veremeyenko wrote:
>> > any objection against introducing *mlt_image_yuv422p10* ?
>> >
>>
>> I am in full support of 422 and planar formats. Just wondering about the
>> bit packing.
>>
>> What would be the corresponding AV format?
>>
>> Would it be 16 bpp with 6 bits unused? BE or LE?
>>
> AV_PIX_FMT_YUV422P10LE
>
>> If it is going to be 16bit samples, then why not call it
>> mlt_image_yuv422p16 and use it for any bit depth between 9 and 16?
>
> introducing *mlt_image_yuv422p16* that equal to AV_PIX_FMT_YUV422P16LE
> (http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavutil/pixdesc.c;h=3b9c45d035f5f2399a652a00b41832f55a8febe1;hb=HEAD#l1496)
> fine to me...

LE makes sense to me. AV_PIX_FMT_YUV422P10LE is the native output of
ffmpeg for H.264 and h.265 10bit decoding. So no endian conversion would
be required when decoding 10bit streams. And Decklink 10bit pixel
packing is so complicated that BE or LE is equally difficult.

I would let Dan make the final call, but it might be good to consider
mlt_image_yuv422p16.

Advantages:
* Allows more support for greater bit-depths in the future.
* Reduces rounding errors when processing <16bit images.
* There is no additional performance cost when processing 12/14/16bit
images in filters.
* Avoids having p10, p12, p14 and p16 formats in the future.
* There would be no additional performance cost for consumers that
do not handle AV_PIX_FMT_YUV422P10LE because a conversion is
required anyway (e.g. decklink 10bit output)

Disadvantages:
* Would require a conversion when a producer is receiving native
AV_PIX_FMT_YUV422P10LE (e.g. decoding 10bit h.264).
* Would require a conversion when a consumer requires native
AV_PIX_FMT_YUV422P10LE (e.g. encoding 10bit h.264)

~Brian
Loading...