Re: [WM]: Evaluation and comparison criteria of watermarking schemes

[Angela Piper <amp08@uow.edu.au>]

Hi, this might be a bit long, there are a lot of things to think about.
;-)

The common practice for evaluating algorithms is to apply them to some images, then measure which performs best on average. The
details depend on what the watermarking algorithms are, where you want to use them, and how thorough you want to be.

If you're comparing robust algorithms, you probably care about resistance to attacks. Measures you might use include the watermark
detection statistic, the bit error rate, or the false positive rate.
Capacity and imperceptibility form a 3-way tradeoff with robustness, so if you're comparing on robustness you need to ensure that
payload and imperceptibility are made equal.

If your application only requires robustness to particular attacks, focus on those. If you need more general robustness, or don't
have a target application, Stirmark (or a similar program) will help you compare using a wide range of attacks. Also consider
whether other properties of the algorithms blind/nonblind will result in a better fit to your application.

Fragile algorithms are similar, but you'll look at whether the watermark is destroyed by image manipulation. Consider both naive
tampering and malicious attacks trying to preserve the watermark. For semi-fragile watermarking you want robustness to some and
fragility to others. For tamper locating algorithms you'll want measure detection sucess/failure for each block; granularity of
tamper resolution is also important.

Finally, I suggest you carefully consider how many images you'll use.
More images give more reliable results (but cost more time). How many images will be needed depends on a lot of factors, a good book
on statistical methods or, even better, consultation with a statistician will help you there. Which images you use also matters, try
to get a good range similar to those for your target application.

Check out:
Kutter and Petitcolas. A fair benchmark for image watermarking systems.

Macq, Dittman and Delp. Benchmarking of image watermarking algorithms for digital rights management.

Fridrich and Goljan. Comparing Robustness of Watermarking Techniques.

Solachidis, Tefas, Nikolaidis, Tsekeridou, Nikolaidis and Pitas.
A benchmarking protocol for watermarking methods.

Pereira, Voloshynovskiy, Madueo, Marchand-Maillet, Pun.
Second Generation Benchmarking and Application Oriented Evaluation.

for more detail on this subject than I can fit in even a long e-mail. 
I hope this helps.

Yours,
Angela Piper




On Thu, Mar 02, 2006 at 12:21:13PM +0100, Djeutanffffffffffe9 Raoul wrote:
> Good mornig every body.
> 
> I have some questions about the watermark. 
> 
> I'm studying some watermarking algorithms and i want to compare them. 
> The problem is that I don't have any criteria which can help me to declare which is better than another.
> 
> So i would want you to help me. I need to know what are the criteria 
> of evaluation of watermarking scheme and what are the comparison criteria of two watermarking schemes.
> 
> Thank You
> 


______________________________________________________________________________

Watermarking Mailing List - http://www.watermarkingworld.org/ml.html
To unsubscribe send email to "majordomo@watermarkingworld.org" with
"unsubscribe watermarking YOURMAIL" in the body.
______________________________________________________________________________