<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Michael Kammes (.com) &#187; Storage</title>
	<atom:link href="http://michaelkammes.com/category/storage/feed/" rel="self" type="application/rss+xml" />
	<link>http://michaelkammes.com</link>
	<description>Full of tech goodness - workflows, tips, tricks, reviews, and insights into the post production realm</description>
	<lastBuildDate>Wed, 11 Jan 2012 15:52:28 +0000</lastBuildDate>
	<language></language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.1</generator>
		<item>
		<title>Choosing the right shared storage solution: Part 2 of 3: Size, Spindles, and Protection</title>
		<link>http://michaelkammes.com/storage/choosing-the-right-shared-storage-solution-part-2-of-3-size-spindles-and-protection/</link>
		<comments>http://michaelkammes.com/storage/choosing-the-right-shared-storage-solution-part-2-of-3-size-spindles-and-protection/#comments</comments>
		<pubDate>Wed, 22 Jun 2011 04:26:40 +0000</pubDate>
		<dc:creator>Michael Kammes</dc:creator>
				<category><![CDATA[NAS]]></category>
		<category><![CDATA[SAN]]></category>
		<category><![CDATA[Storage]]></category>

		<guid isPermaLink="false">http://michaelkammes.com/?p=1487</guid>
		<description><![CDATA[This is Part 2 in a 3 part series on Choosing the Right Shared Storage Solution.  You may want to check out Part 1: Bandwidth &#38; Connections. Hard drives are a dime a dozen nowadays. At last check, I think Best Buy and Fry&#8217;s had them at the checkout counter next to the latest Star Magazine and Chewlies gum. Despite the seemingly over abundance of drives, not all drives (let alone a collection of &#8216;em) are created equal. Most hard drives, as I&#8217;m sure you know, have spinning platters in them were data resides. While spinning, they allow data to ...]]></description>
			<content:encoded><![CDATA[<p><em>This is Part 2 in a 3 part series on Choosing the Right Shared Storage Solution.  You may want to check out <a href="http://michaelkammes.com/storage/choosing-the-right-shared-storage-solution-part-1-of-3-bandwidth-connections/" target="_blank">Part 1: Bandwidth &amp; Connections.</a></em></p>
<p>Hard drives are a dime a dozen nowadays.  At last check, I think Best Buy and Fry&#8217;s had them at the checkout counter next to the latest Star Magazine and Chewlies gum.  Despite the seemingly over abundance of drives, not all drives (let alone a collection of &#8216;em) are created equal.</p>
<p>Most hard drives, as I&#8217;m sure you know, have spinning platters in them were data resides.  While spinning, they allow data to be read and written to them.  The faster the platter (spindle) the quicker a computer can read and write the data.  So, wouldn&#8217;t the same hold true if you connect a bunch of drives (hence the storage geek acronym: JBOD: Just A Bunch of Drives) together?  Why yes, you&#8217;ve got a point there.  More drives = more and faster throughput.</p>
<p>So, a JBOD, with some management software on it, and <em>WHAMMO</em>, you&#8217;ve got a large mass of faster, useable storage.  Should be pretty cheap, right?</p>
<h4><em><strong>Wrong.</strong></em></h4>
<p>Let&#8217;s start at the beginning where this logic falls apart.</p>
<p>All drives are not created equal.  Drives are measured in many categories.  Some of which include:</p>
<p><strong>RPM </strong>(Spindle speed)<br />
Essentially, the faster the better.  The faster the rotational speed, the faster the drive can be written and read to, barring no other bottlenecks.</p>
<p><strong>Disk Buffer: </strong><em>(AKA Disk Cache or Cache Buffer)<br />
</em> This is where data is stored on the drive temporarily, before it&#8217;s read from or written to the drive.  This allows time for the disk to &#8220;catch up&#8221; if the requests for reading and writing cannot be met immediately.   However, it mainly makes the reading to and writing from a drive more efficient and organized.  Bigger is usually better (currently, most drives are 8-32MB in size), but there is some debate over the validity to this point. However, since Enterprise drives typically have bigger Cache/Buffers, take the MBs and run.</p>
<p><strong>Interface </strong><em>(SAS, SATA, PATA, IDE, SCSI, TIN CAN AND STRING)</em><br />
Each drive I/O interface has it&#8217;s own quirks and own thresholds, but for our discussion, it&#8217;s main limiting factor is how much data the connection allows to flow through it at one time.  SATA or SAS are the most common interface nowadays for single drives or small arrays.  These interface types typically allow for more throughput than single drive could ever deliver.  Thus, a SAS or SATA connection is rarely your bottleneck, until you get into many &#8220;striped&#8221; drives (an array).  Once you get into this realm, we move to other, more robust solutions.  However, as we discussed in <a href="http://michaelkammes.com/storage/choosing-the-right-shared-storage-solution-part-1-of-3-bandwidth-connections/" target="_blank">Part 1 of Choosing the right shared storage solution: Bandwidth &amp; Connections</a>, a fatter pipeline to and from the array may NOT be what you need.</p>
<p><strong>MTBF </strong><em>(Mean Time Between Failure)</em><br />
Muy importante. The lower the MTBF, the less robust the drive is, compared to others with &#8220;enterprise&#8221; branding.  This means the drive may fail earlier in its lifetime of expected use, compared to an enterprise-class drive with a higher MTBF.  This also comes at a price premium &#8211; better parts and more strict QC.  If you want the steak, you gotta pay for it, lest you get ground chuck.</p>
<p><strong>Size</strong><br />
The bigger the better, ladies.  More storage = more space for your important media professionally-ripped for marketing campaigns, and pr0ntube, err, research.</p>
<h3><strong>Shades of Drives</strong></h3>
<p>Some manufacturers market their drives into more easily to digest categories, as hard drives can have varying degrees of quality components.  Let&#8217;s take Western Digital&#8217;s drive categorization to better illustrate this.  WD has<span style="color: green;"><strong>Green</strong></span>, <span style="color: blue;"><strong>Blue</strong></span>, and <strong>Black</strong> designations.  <span style="color: green;"><strong>Green</strong></span> drives are usually in the mid to slower side of the the speed pool, and hey also tend to spin at lower RPMs and have smaller disk cache&#8217;s.  They are called <span style="color: green;"><strong>Green</strong></span> because they tend to spin down when not in use. This saves energy, saves money, and saves Gaia.  These are usually the least expensive drives.  They are horrible for video usage, as how can you playback fat video files if a drives decides to take a nap?</p>
<p><span style="color: blue;"><strong>Blue</strong></span> drives are more meant for everyday computing and are commonly found in laptops.  They are usually priced around the same, if not slightly more, than <span style="color: green;"><strong>Green</strong></span> drives.  These are the most common of the HDDs out there, and are mid range on the speed race.  <strong>Black</strong> drives are enterprise class, are usually the fastest, and have the lowest MTBF.  They can operate in warmer temperatures (drive arrays can get toasty) and due to these extras, are the most expensive in the bunch.  While I use Western Digital as an example, drives with specs which are similar to WD&#8217;s <strong>Black</strong> classification are what you WANT in a mass storage solution.  These are not your fathers hard drives, are typically NOT the ones you see in the weekly electronics flyer.  You need to seek them out.</p>
<p>As we&#8217;ll examine later, determining the right combination of the right hard drives, then building software around them is paramount to decent performance.</p>
<h3><strong>Assemble the troops</strong></h3>
<p>That was just choosing the drives.  That, in and of itself, is a pain.  Now, we need a chassis to house the drives. When you buy a shared storage <em>SOLUTION</em>, the solution provider (manufacturer or vendor) has already factored in all of these drives variables and incorporated a drive chassis in order to create a turnkey package *just* for you.</p>
<p>When designing these turnkey solutions of drives and chassis, typically the Manufacturer or Vendor will:</p>
<ul>
<li>build or OEM a chassis which holds the drives, and test the aforementioned drives IN the chassis to assure performance is good and constant</li>
<li>beat up on the drives for failure and performance with various software applications</li>
<li>systematically write data across the drives equally to ensure sustained performance as the drives fill up.</li>
<li>design management and sharing software for the user to actually USE the data, and possibly with other users.  I will cover this in detail in part 3 of the 3 part shared storage blog.</li>
<li>write software to poll the drives to check for impending failures (health), as well as optimize the usage of their buffer(s)</li>
<li>last but most critical, protect the data (using RAID or better technology)</li>
</ul>
<p>These solutions, if done right, relay on a battle tested combination of hardware components.  Off the shelf components, put together because the cables fit, will never deliver the performance a tuned system can. This is yet another reason why consulting a shared storage systems integrator or consultant assures you&#8217;re getting a best of breed solution &#8211; not product scattershot.  <em>I&#8217;m serious about this and I can&#8217;t stress this point enough.</em></p>
<p><em>I know, this is fun, right?  Are you not entertained?!</em></p>
<p>We now move on to yet another acronym: <strong>RAID</strong>.</p>
<p><em>RAID, RAID, RAID. Oh how you complicate thee.</em></p>
<p>A RAID is an Redundant Array of Independent Disks.  Take this scenario: suppose a drive fails (MTBF) once in 1 million uses.  We don&#8217;t know *when* it will happen, we just know it probably will &#8211; and before 1 million uses.  Where this happens is up to chance, environment, and usage.  Now, let&#8217;s say we RAID two drives together as one, because that would yield twice as much space and speed.  This obviously increases the risk of MTBF.  Plus, if ONE (yes, ONE) drive starts to smoke &#8211; you&#8217;ve lost all of your data, because you spanned the disks together.  You can&#8217;t edit with half of every bit and byte gone.  Given this truth, now multiply this by 4 drives.  Howabout 16 drives or more? Russian Roulette, geek-style. Combining drives in this manner is known as RAID 0, the largest combination of throughput and capacity, but absolutely no support for preserving data if one drive dies. This absolutely bites when it comes to <em>DATA AVAILABILITY</em>.</p>
<p>The loss of one drive in a RAID 0 array could be a massive problem for the video editor (you just lost the entire movie, no problem&#8230; right?!?).  Since this is not acceptable in most circumstances, other RAID data-protection and performance formats have been developed that ensure there is some <em>REDUNDANT </em>distribution of data across multiple disks. (Ah yes, the<strong> R</strong> in <strong>RAID</strong>!) While there are as many as the day is long, let&#8217;s examine those you will probably find out in the wild when dealing with video shared storage solutions:</p>
<p><strong>RAID 1</strong>: If RAID 0 doubles your storage, then conversely,  RAID 1 cuts the cumulative size in half.  Why?  RAID 1 (AKA &#8220;mirroring&#8221;) allows that in the event of the rapture and half of all of the drives in your array blow chunks, that you don&#8217;t lose any of your data, because a 1:1 copy of the data has been made on that extra storage.  This can cause a slight hit in throughput (after all, your data is being written twice), and as outlined, a massive hit in storage space.  Those of you who have used Avid&#8217;s Unity (not ISIS) have used RAID 0 or RAID 1 for years &#8211; it&#8217;s all Unity supported.  Most other shared storage solutions also offer RAID 0 or RAID 1 as well.</p>
<p><strong>RAID 2</strong>,<strong> 3</strong>, &amp; <strong>4</strong>.  Outdated, or yielded unnecessary by RAID 5.  Move along.  <em>BTW, what ever happened to Leonard Parts 1-5?</em></p>
<p><a href="http://michaelkammes.com/wp-content/uploads/2011/06/raid5-slogan.png"><img class="size-full wp-image-1489 alignright" title="RAID5 - Kills Data Loss Dead" src="http://michaelkammes.com/wp-content/uploads/2011/06/raid5-slogan.png" alt="RAID5 - Kills Data Loss Dead" width="225" height="290" /></a></p>
<p><strong>RAID 5</strong>: Probably the most popular out there.  It balances throughput and redundancy, with minimal overhead. This achieved through parity. (<em><strong>Warning! </strong>Higher geek content:</em> When data is written, parity data is introduced and written with your video data across the drives. In the event a drive fails, this parity data is combined with existing media to recreate the lost media for your enjoyment.  As one can imagine, this decreases performance slightly: not only while the initial parity data is created and written, but if a drive dies, the array needs to recreate the media for usage in real time.)  All this being said, RAID 5 allows for the speed benefit afforded by a RAID, along with good redundancy.  As a bonus, if a drive dies, most shared storage chassis can rebuild the lost media once a correct drive is inserted into the chassis to replace the dead drive &#8211; restoring your array to it&#8217;s former glory.  Just give it until tomorrow, it usually takes a bit of time and you will see a modest performance hit during the rebuild process. But hey, it&#8217;s not <em>gone!</em></p>
<p><strong>RAID 6</strong>: Very similar to RAID 5, although the user has one more guard at the gate: 2 drives worth of parity are written, instead of 1.  If a drive does die, there remains 1 drive still spinning and keeping the array functional.  Same basic performance and storage hits as RAID 5.  RAID 6 is slightly less common when seeking out Video RAIDs.</p>
<p>I usually ballpark a 12-20% hit on storage space AND throughput for your shared storage solution to handle a RAID5. This varies by manufacturer, but the 12-20% plays heavily into my storage formula at the end of this article. I know;  no one wants to lose space, but it&#8217;s better than losing half of your space with RAID1 or having no redundancy, a la RAID 0.</p>
<p>Other less popular RAID formats include RAID 6, RAID 0+1, RAID1+0 (AKA RAID 10), RAID 0+3, RAID 3+0, etc.  Consult your local storage geek if you *realllllly* want to delve into these.</p>
<p>It should be noted that a RAIDSET can be done at either the hardware level, or the software level.  In Windows or on OS X, for example, you can RAID drives (usually RAID 0 or 1, rarely RAID 5) from within the OS. (Manage&#8211;&gt;Storage and Disk Utility, respectively)   This is a software RAID, and while it works, it&#8217;s usually not as fast or bullet-proof as a hardware RAID 0 or 1, which (if available) is done on the chassis which contains your drives.  Most hardware RAID controllers are designed specifically for RAID 1, 5 and 6. Hardware RAID is faster than software RAID for managing the layout of data and parity bits.</p>
<p><em><strong>Now for my patent pending formula.</strong></em></p>
<p>Let&#8217;s take a 1 TB drive.  Small, and easy on my math challenged brain.</p>
<p>As you probably know, marketing 1TB is not equal to 1TB useable storage.  <em>This hard drive loss is due to base 2 math rather than base 8. (1000 bytes = 1 kilobyte marketing, 1024 bytes = 1 kilobyte in reality)  (Editor&#8217;s note: Thanks Micheal!)</em> Thus, when you begin to multiply bits, bytes, kilobytes, megabytes, gigabytes, etc…you end up with less than 1TB.  And of course, marketing wins: 1TB is easier to sell than 930GB.  So, we&#8217;re saddled with a 7% loss.  Keep that number written down.</p>
<p>We now need to initialize the drives, and format them into a RAID.  Let&#8217;s say we go with RAID 5.  Best balance of speed and redundancy.  RAID 5 in hardware can be between 12-20% in loss of space due to the aforementioned redundancy.   Again, this is different for each manufacturer, so no need for the math hate mail.  Let&#8217;s use 15%, and subtract that from 930GB.  This comes out to approx. 790GB.  So, now we&#8217;re down 210GB from the advertised size.</p>
<p>As I mentioned earlier, performance (throughput, in this case) can decrease as the drive fills up, if the data is written sequentially on the disc(s).  Think of trying to pull a toy from the bottom of a box of cereal: it&#8217;s tougher to get to when the box is full of cereal than when it&#8217;s empty. Some shared storage manufacturers (Facilis comes to mind) scatters the data around the drive, so a user never sees a performance hit;  as performance is equal regardless of the amount of free space.   That&#8217;s in the minority, so the magic number before a noticeable loss in throughput seems to be around 80%.  Thus, we subtract another 20% from the 790GB.  This comes out to 632GB.</p>
<p>That&#8217;s right.  Of that shiny new 1TB drive, once introduced into a RAID 5 RAIDSET, and given some room for performance, we have nearly a 40% space loss.</p>
<p style="text-align: center;"><a href="http://michaelkammes.com/wp-content/uploads/2011/06/hard-drive-gauge.png"><img class="size-full wp-image-1488 aligncenter" title="Useable Storage Space vs. Advertised Storage Space" src="http://michaelkammes.com/wp-content/uploads/2011/06/hard-drive-gauge.png" alt="Useable Storage Space vs. Advertised Storage Space" width="550" height="315" /></a></p>
<p>So concludes Part 2 of our 3 part series.  Stay tuned for Part 3: Management, Permissions &amp; Support.  Same bat time, same bat channel.</p>
<p><em>Special Thanks: <a href="http://www.flickr.com/photos/michaelkammes/5617421981/" target="blank">David Sallak</a> of <a href="http://www.isilon.com/" target="blank">Isilon.</a></em></p>
]]></content:encoded>
			<wfw:commentRss>http://michaelkammes.com/storage/choosing-the-right-shared-storage-solution-part-2-of-3-size-spindles-and-protection/feed/</wfw:commentRss>
		<slash:comments>3</slash:comments>
		</item>
		<item>
		<title>Choosing the right shared storage solution: Part 1 of 3: Bandwidth &amp; Connections</title>
		<link>http://michaelkammes.com/storage/choosing-the-right-shared-storage-solution-part-1-of-3-bandwidth-connections/</link>
		<comments>http://michaelkammes.com/storage/choosing-the-right-shared-storage-solution-part-1-of-3-bandwidth-connections/#comments</comments>
		<pubDate>Tue, 07 Jun 2011 13:54:50 +0000</pubDate>
		<dc:creator>Michael Kammes</dc:creator>
				<category><![CDATA[NAS]]></category>
		<category><![CDATA[SAN]]></category>
		<category><![CDATA[Storage]]></category>
		<category><![CDATA[workflow]]></category>

		<guid isPermaLink="false">http://michaelkammes.com/?p=1356</guid>
		<description><![CDATA[Shared Storage – the least sexy of the post production technologies. Shared Storage is not the hot chick in class. She&#8217;s the quiet librarian with glasses that you end up falling for when the unreliable hot chick loses her sheen. She&#8217;s got everything you need: reliability and stability, and flashing lights. (Maybe the flashing lights thing is just me.) In part 1 of 3 part posting, I&#8217;m going to examine shared storage for video – SAN &#38; NAS. Many factors go into choosing the correct shared storage solution: Bandwidth &#38; Connections Drives: Size, Spindles &#38; Protection (RAID) Management, Permissions &#38; ...]]></description>
			<content:encoded><![CDATA[<p>Shared Storage – the least sexy of the post production technologies.  Shared Storage is not the hot chick in class.  She&#8217;s the quiet librarian with glasses that you end up falling for when the unreliable hot chick loses her sheen.  She&#8217;s got everything you need: reliability and stability, and flashing lights.  (Maybe the flashing lights thing is just me.)</p>
<p>In part 1 of 3 part posting, I&#8217;m going to examine shared storage for video – SAN &amp; NAS.</p>
<p>Many factors go into choosing the correct shared storage solution:</p>
<ol>
<li>Bandwidth &amp; Connections</li>
<li>Drives: Size, Spindles &amp; 	Protection (RAID)</li>
<li>Management, Permissions &amp; Support</li>
</ol>
<p>In this post, we&#8217;ll examine #1: Connections and Bandwidth.</p>
<p>Before we jump into the geeky goodness, let&#8217;s make sure we&#8217;re all following the same nomenclature.</p>
<ol>
<li><strong>SAN:</strong> Storage Area Network.  Based on Fibre topology.</li>
<li><strong>NAS:</strong> Network Attached Storage. Based on Ethernet topology.</li>
<p>Currently, there exists 2 main methods of connecting many computers to the same storage: Ethernet and Fibre.  A good rule of thumb is if it&#8217;s Fibre, it&#8217;s a SAN, and if it&#8217;s Ethernet, it&#8217;s a NAS.  While there are a few exceptions, this guideline is true a <em>majority</em> of the time. <em>I humbly await the flames&#8230;</em></p>
<p>The endgame is that BOTH (when configured and deployed correctly) can be run long distances, are somewhat flexible, and most importantly, can be used for mass storage in the video realm.  The big difference to the end user is <em>bandwidth</em>.</p>
<p>Bandwidth &#8211; also known as throughput &#8211; is the amount of constant data your computer can sustain to and from storage, and is paramount for video playback.  Too little bandwidth &#8211; or poor sustained bandwidth &#8211; causes your video to stutter or downright refuse to play.</p>
<p>When it comes to bandwidth, the video industry is actually in an excellent place right now.  Acquisition codecs, like h.264 (Canon 5D, 7D),  XDCAM, and P2, take up a small amount of bandwidth, while still delivering decent quality.  Let&#8217;s look at Standard Definition (SD) video from 10 years ago.  Many broadcast networks would play material to air that had been onlined at 1:1 uncompressed or frequently 2:1.  The video datarate of SD at  1:1 is 22 megabytes (MB) per second.  The current broadcast quality HD Avid codec (DNxHD 145) is actually <em><strong>less</strong></em>, clocking in at about 18MB/s.  Heck, even DNxHD 220x (the least amount of compression before going uncompressed in the Avid realm) is only about 27 MB/s.  Thus, we can take advantage of the drops in storage pricing, advancement of computing power and still use the same bandwidth we were using back in the SD days.</p>
<p>Below is a quick list of <em>current</em> Ethernet and Fibre transmission speeds.</p>
<table border="1" cellspacing="0" cellpadding="4" width="404">
<colgroup>
<col width="193"></col>
<col width="193"></col>
<col width="193"></col>
</colgroup>
<tbody>
<tr valign="TOP">
<td width="193" height="16"><strong>Connection</strong></td>
<td width="193"><strong>Throughput</strong></td>
<td width="193"><strong>20% Real World</strong></td>
</tr>
<tr valign="TOP">
<td width="193" height="17">4GB Fibre</td>
<td width="193">425MB/s</td>
<td width="193">~340 MB/s</td>
</tr>
<tr valign="TOP">
<td width="193" height="17">8GB Fibre</td>
<td width="193">850 MB/s</td>
<td width="193">~680MB/s</td>
</tr>
<tr valign="TOP">
<td width="193" height="17">1 Gig Ethernet</td>
<td width="193">125 MB/s</td>
<td width="193">~100 MB/s</td>
</tr>
<tr valign="TOP">
<td width="193" height="16">10Gig Ethernet</td>
<td width="193">1250 MB/s</td>
<td width="193">~1000 MB/s</td>
</tr>
</tbody>
</table>
<p>The smart reader has noticed that <em>real world</em> throughput differs from <em>theoretical</em> throughput.  For example, Ethernet typically has <strong>more</strong> overhead (read: less bandwidth) than Fibre, but neither typically perform as fast as the <em>theoretical</em> speed.  Kind of like that bag of life-shortening potato chips you picked up at the grocery store.  It&#8217;s a bag of chips, yes, but is it ever really filled to the top?</p>
<p>Personally, I normally ballpark 10-30% off of theoretical for a more real world ballpark estimation.  <strong>ALL </strong>shared storage solutions are a little bit different.  However, for sake of discussion, and to be safe when planning your facility, let&#8217;s say 20%.</p>
<p>In addition, all of these connections inherently have the ability to be <em>bonded</em>, that is, the aggregate speed of multiple connections yield more throughput.  This, of course, is based on the SAN or NAS, the switch, and the Fibre or Ethernet card in your system supporting it.  Never assume any of them handle aggregation– verify.  If they do, then you now have an opportunity to potentially double your available bandwidth.  This is a very good thing.</p>
<p>Now that we know how wide our highway is, let&#8217;s see how many cars we can fit on it.  Here is a short list of some of the most popular codecs at their highest data rates.</p>
<table border="1" cellspacing="0" cellpadding="4" width="404">
<colgroup>
<col width="137*"></col>
<col width="119*"></col>
</colgroup>
<tbody>
<tr valign="TOP">
<td width="53%"><strong>Codec (at 1080i/60)</strong></td>
<td width="47%"><strong>Speed (Mb/s &amp; MB/s)</strong></td>
</tr>
<tr valign="TOP">
<td width="53%">AVCHD</td>
<td width="47%">24 Mb/s (3 MB/s)</td>
</tr>
<tr valign="TOP">
<td width="53%">HDV</td>
<td width="47%">25Mb/s  (3.125 MB/s)</td>
</tr>
<tr valign="TOP">
<td width="53%">Canon7D (h.264) (1080p)</td>
<td width="47%">48 Mb/s  (6MB/s)</td>
</tr>
<tr valign="TOP">
<td width="53%">XDCAM HD 50</td>
<td width="47%">50Mb/s  (6.25 MB/s)</td>
</tr>
<tr valign="TOP">
<td width="53%">AVC-Intra (100)</td>
<td width="47%">100 Mb/s  (12.5 MB/s)</td>
</tr>
<tr valign="TOP">
<td width="53%">DVCPro HD</td>
<td width="47%">115 Mb/s  (14.4 MB/s)</td>
</tr>
<tr valign="TOP">
<td width="53%">DNxHD 145</td>
<td width="47%">145 Mb/s  (18.125 MB/s)</td>
</tr>
<tr valign="TOP">
<td width="53%">ProRes 422 (HD)</td>
<td width="47%">167 Mb/s  (20.82 MB/s)</td>
</tr>
<tr valign="TOP">
<td width="53%">REDCODE 42</td>
<td width="47%">336 Mb/s  (42 MB/s)</td>
</tr>
<tr valign="TOP">
<td width="53%">Uncompressed 10-bit RGB</td>
<td width="47%">1990.24Mb/s  (248.78 MB/s)</td>
</tr>
</tbody>
</table>
<p>For you non-math majors (don&#8217;t worry, I&#8217;m not one, either) you&#8217;ll notice some big number discrepancy.  Here&#8217;s why:</p>
<p>Apparently, not everyone got the memo on standardizing measuring practices.  So much so, that codec data rates are typically measured in Mega<em>BITS</em> (often written as Mb/s, Mbit/s, or Mbps), while storage is usually calculated in Mega<em>BYTES</em> (MB/s), Giga<em>BYTES</em> (GB/s), <em>Tera</em>BYTES, (TB/s) etc.  This makes calculating storage somewhat of a headache.  A tip?  Divide the codec data rate by 8.  (A byte contains 8 bits).  Thus, a codec running at 48 Mb/s = 6MB/s.  <em>Genius</em>.</p>
<p>So, let&#8217;s now match up our cars (codecs) against our highway (throughput).  Below we see codecs vs bandwidth <em>(remember, bandwidth calculated at 20% less than theoretical max, which is a better approximation of real-world performance):</em></p>
<div id="attachment_1474" class="wp-caption aligncenter" style="width: 610px"><a href="http://michaelkammes.com/wp-content/uploads/2011/06/codec-and-bandwidth-600.png"><img class="size-full wp-image-1474" title="Codecs and Bandwidth Comparison" src="http://michaelkammes.com/wp-content/uploads/2011/06/codec-and-bandwidth-600.png" alt="Codecs and Bandwidth Comparison" width="600" height="529" /></a><p class="wp-caption-text">Codecs and Bandwidth Comparison</p></div>
<p>So, what does this mean?</p>
<p>Clearly, most codecs can easily fit through the pipeline 1GigE provides.  Fibre becomes the obvious solution when uncompressed codecs (or similiar) come into play.  Thus, for most video-centric applications, an Ethernet connected solution &#8211; single or bonded (even better) &#8211; can handle the job.</p>
<p>I would add that not all Ethernet solutions are created equal.  Sustained throughput is the name of the game, and off-the-shelf or I.T. centric solutions almost never handle video properly.  A Pinto can get onto a race track, doesn&#8217;t mean it performs well, ya know?</p>
<p>In closing for the 1st entry in the 3 part series, (1 for 3 may be good in baseball, but not in Post, and certainly not when choosing a shared storage system) I cannot stress enough that if you&#8217;re considering a SAN or NAS, consult a <strong>Video</strong> Shared Storage specialist.  SAN and NAS solutions are not at the point where they are plug and play.</p>
<p><em>Yet.</em></p>
<p>Be on the look out for part 2: Drives: <em>Size, Spindles &amp; Protection (RAID)</em>.</p>
<p><em>Special Thanks: <a href="http://www.flickr.com/photos/michaelkammes/5617421981/" target="blank">David Sallak</a> of <a href="http://www.isilon.com/" target="blank">Isilon</a>, <a href="http://www.keycodemedia.com" target="blank">Pat Howley of Key Code Media</a>.</em></ol>
]]></content:encoded>
			<wfw:commentRss>http://michaelkammes.com/storage/choosing-the-right-shared-storage-solution-part-1-of-3-bandwidth-connections/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>It came from the Searches Volume 1</title>
		<link>http://michaelkammes.com/editorial/it-came-from-the-searches-volume-1/</link>
		<comments>http://michaelkammes.com/editorial/it-came-from-the-searches-volume-1/#comments</comments>
		<pubDate>Fri, 26 Feb 2010 19:01:14 +0000</pubDate>
		<dc:creator>Michael Kammes</dc:creator>
				<category><![CDATA[Avid]]></category>
		<category><![CDATA[Digital Asset Management]]></category>
		<category><![CDATA[Editorial]]></category>
		<category><![CDATA[Encoding]]></category>
		<category><![CDATA[Searches]]></category>
		<category><![CDATA[Storage]]></category>
		<category><![CDATA[Tech Note]]></category>
		<category><![CDATA[AJA]]></category>
		<category><![CDATA[codec]]></category>
		<category><![CDATA[Final Cut Server]]></category>
		<category><![CDATA[RED]]></category>
		<category><![CDATA[Sonnet]]></category>
		<category><![CDATA[Telestream]]></category>

		<guid isPermaLink="false">http://michaelkammes.com/?p=609</guid>
		<description><![CDATA[I&#8217;m on a plane from Vegas &#8211; after nerding out at the Digital Signage Expo.  It&#8217;s a full circle, having been christened into Digital Signage around 2002 with Graybow.  Glasfire (3M&#8217;s Vikuiti), if any of you remember.  In any event, this short 50 minute jaunt from Vegas to Burbank gives me time to write a quick blog. Matt Stratton turned me on to Clicky Stats awhile ago, and through Clicky, I&#8217;m able to see what web searches come up with my blog as a result. I also see when these searches lead them to my blog &#8211; but then they ...]]></description>
			<content:encoded><![CDATA[<p>I&#8217;m on a plane from Vegas &#8211; after nerding out at the <a href="http://www.digitalsignageexpo.net/" target="_blank">Digital Signage Expo</a>.  It&#8217;s a full circle, having been christened into Digital Signage around 2002 with <a href="http://www.graybow.com/" target="_blank">Graybow</a>.  <a href="http://www.graybow.com/product_sales/glasfire.html" target="_blank">Glasfire</a> (<a href="http://solutions.3m.com/wps/portal/3M/en_US/VikuitiHome/Landing-Page/" target="_blank">3M&#8217;s Vikuiti</a>), if any of you remember.  In any event, this short 50 minute jaunt from Vegas to Burbank gives me time to write a quick blog.</p>
<p><a href="http://mattstratton.com">Matt Stratton</a> turned me on to <a href="http://getclicky.com/" target="blank">Clicky Stats</a> awhile ago, and through Clicky, I&#8217;m able to see what web searches come up with my blog as a result.  I also see when these searches lead them to my blog &#8211; but then they leave because they didn&#8217;t find the answer.  I thought since the search engines think I already have it on my site, perhaps I should.    Below is a sampling of the search queries that my website supposedly already has the answers for.  So, I present to you:</p>
<h3><strong>It came from the Searches, Volume 1</strong>.</h3>
<p><strong>dnx-36 firewire 400</strong> and <strong>bandwidth requirements DNX36</strong><br />
Yes, DNxHD36  (aka DNx36) can certainly work through a firewire400 pipe.  DNxHD36 runs at a 36mbps, while firewire400 has a theoretical max of 400mbps.  This makes firewire400 a fine choice as a transport for DNx36 offline workflows.</p>
<p><strong>29.97 and dnx36</strong><br />
This is a source of confusion for many Avid users, and I&#8217;m in the process of authoring a blog on this.  DNxHD36 is a &#8220;family&#8221; according to Avid.  While the &#8220;36&#8243; label refers to the data rate for 23.976 material, the DNxHD36 &#8220;family&#8221; codec can be applied to 29.97 material.  This would be technically known as DNxHD45 (45mbps as opposed to 36mbps).</p>
<p><strong>&#8220;episode engine&#8221; cpu restriction</strong><br />
This is kind of vague.  Within Episode, you can assign priority of encodes and cpu usage.  Quicktime already is pretty poor with processor usage to begin with, however.</p>
<p><strong>Metafuze Lossy?</strong><br />
Yes.  After all, you&#8217;re transcoding from one codec to another &#8211; there will always be quality loss.  In addition, DNxHD is a lossy codec &#8211; so, yes.</p>
<p><strong>edit room day rate</strong><br />
This varies wildly.  Depends what gear is in the room, what the room is used for (<em>editorial, audio, finishing, color, etc.</em>) your experience, and geographical location &#8211; and hell, what your rent is! Normally a reputable rental rate at a good facility would be a couple hundred bucks per hour or even much more.   As a side note / soap box,  don&#8217;t sell out the rest of the industry by undercutting other facilities.  Talent and experience are worth the money.  That&#8217;s a majority of what the client is paying for &#8211; not the gear.  By undercutting everyone else, the industry is forced to adopt your lower rate as a standard, then everyone has to drop their prices.  And the cycle continues. You&#8217;re shooting yourself in the foot.</p>
<p><strong>Does clipster support facilis</strong><br />
Clipster runs on Windows XP 64bit, which Facilis has a client for.  Fibre is the best choice (Ethernet won&#8217;t deliver the bandwidth needed), and it should be a recent version of Facilis software to enable file locking &#8211; volume locking will cause organization issues, but can be done.</p>
<p><strong>convert red footage cluster setup</strong><br />
I wish I could do it better than these guys have:<br />
<a href="http://www.kenstone.net/fcp_homepage/compressor_multi_cores_stitzer.html" target="blank">http://www.kenstone.net/fcp_homepage/compressor_multi_cores_stitzer.html</a><br />
<a href="http://www.fcsoutlet.com/home/Studio_Outlet/Entries/2008/9/1_Virtual_Clusters_-_Compressor.html" target="blank">http://www.fcsoutlet.com/home/Studio_Outlet/Entries/2008/9/1_Virtual_Clusters_-_Compressor.html</a></p>
<p><strong>fcserver script</strong><br />
Good luck. Most everything is manual at this point. Try <a href="http://www.squarebox.co.uk/" target="_blank">CATDV</a> as a more GUI based alternative.</p>
<p><strong>QIO-PCIE sonnet tech</strong><br />
The QIO comes with either a PCIE host card or a E34 (Express slot) host card.  Unveiled at NAB 2009, and finally shipping now!  Mac only support for now.  List Price is about $1000.</p>
<p><strong>FTP, KiPro</strong><br />
Well, once the KiPro encodes to ProRes, you can certainly FTP the file &#8211; although it&#8217;s gonna be huge.  Your best bet is to offload it to a portable drive for transport or use a transfer solution like <a href="http://www.storagedna.com/" target="_blank">StorageDNA</a>.</p>
<p><strong>share msm database avid</strong><br />
I can only assume you&#8217;d like two or more machines to access the same Avid Database at the same time, in order to share media.  Nope &#8211; the functionality you are attempting to do is achieved through Avid Unity, ISIS, and Interplay.</p>
<p><strong>final cut server frontend</strong><br />
Final Cut Server (FCSvr) clients have a front-end based in Java; it looks the same on Mac or PC &#8211; and the administrator can restrict what functions a client can have.  A popular alternative is creating a web page interface which ties into the database and media of FCSvr.  The functions on thew web page are usually limited and typically only used for &#8220;review and approval&#8221; functionality.  Several companies out there can create a custom web based interface &#8211; although it is far from cheap.</p>
<p><strong>episode pro pro res 422</strong><br />
Yes, If Episode is running on a Mac, you can encode and decode into any of the flavors of &#8220;ProRes&#8221; (1 word).  Decoding is only available (currently) on the Windows version.</p>
]]></content:encoded>
			<wfw:commentRss>http://michaelkammes.com/editorial/it-came-from-the-searches-volume-1/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>A little somethin’ for the Producers….mass review and approval made simple.</title>
		<link>http://michaelkammes.com/editorial/a-little-somethin-for-the-producers-mass-review-and-approval-made-simple/</link>
		<comments>http://michaelkammes.com/editorial/a-little-somethin-for-the-producers-mass-review-and-approval-made-simple/#comments</comments>
		<pubDate>Thu, 14 May 2009 17:27:24 +0000</pubDate>
		<dc:creator>Michael Kammes</dc:creator>
				<category><![CDATA[Editorial]]></category>
		<category><![CDATA[Storage]]></category>
		<category><![CDATA[Apple]]></category>

		<guid isPermaLink="false">http://michaelkammes.com/column/?p=222</guid>
		<description><![CDATA[A little somethin’ for the Producers….mass review and approval made simple.]]></description>
			<content:encoded><![CDATA[<p>I recently was posed with a relatively simple quandary from a studio:</p>
<p>How can I get edits which need review out to non tech savvy producers for approval so they can:<br />
A) easily view the edit (did I mention easily?)<br />
B) stay out of the edit room.</p>
<p>Of course, sometimes A) means burning a DVD or laying off a tape, which is time consuming, not secure, and let’s face it, decidedly not green.  It also can eliminate FTPing a file to a server to then be downloaded and watched as a viable solution, as sometimes this walking and talking at the same time complication can cause more time in tech support than just letting them in the edit room, which violates B).</p>
<p>Aside from my <a href="/digital-asset-management/final-cut-server-and-you-overview-workflows-pluses-pitfalls/" target="blank">column and proposed workflows with Final Cut Server</a>, I have something even simpler…and in 2 flavors.</p>
<p>Scenario 1:  Small scale.</p>
<p>What you need:</p>
<p>Your Edit Bay NLE<br />
Generic Computer (Mac, most likely)<br />
AppleTV with Video Monitor (HDMI)<br />
Extras: Encoding solution, Shared storage, same LAN access.</p>
<p>Procedure:  Editor exports a full QuickTime file in the appropriate H.264 format for AppleTV, to a shared network drive.  AppleTV runs off of iTunes, so your generic computer picks up the H.264 file, which sync (or streams) to your AppleTV.  Now, the producer can view the cut in the comfort of the plush couch of his 4 wall, and not in the edit room.  This can be streamlined even further by having the editor export a QT Reference to a watch folder, which the encoding solution would pickup (make sure all volumes containing the referenced media can be seen by the encoder) and proceed with the H.264 encode, then sending the file to the generic computer running iTunes.  Having all of the satellite AppleTVs stream from the 1 machine makes media management very easy.</p>
<p>Limitations?</p>
<p>5 AppleTVs can sync to 1 iTunes Library.<br />
1 AppleTV can stream from up to 5 computers.<br />
AppleTV is specific in what formats it will play.</p>
<p>Scenario 2:  Larger Scale</p>
<p>What you need:<br />
Your Edit Bay NLE<br />
Several Mac Minis with DVI to HDMI cables (for a Plasma / LCD display), Apple Remote.<br />
Extras: Encoding solution, Shared storage, same LAN access.</p>
<p>Same concept applies: Editor exports a QT or QT reference, and either the 3rd party encoding solution delivers the file to a shared network drive, or the NLE does.</p>
<p>Here is where the Mac Mini comes into play.</p>
<p>OS access is a no-no for a 2 year old.  And for Producers – for the same reason.  So, here is what I did:</p>
<div id="attachment_223" class="wp-caption alignright" style="width: 310px"><a href="http://michaelkammes.com/wp-content/uploads/2009/05/parental_controls.gif"><img class="size-medium wp-image-223  " title="Parental Controls Preference Pane" src="http://michaelkammes.com/wp-content/uploads/2009/05/parental_controls-300x292.gif" alt="Parental Controls Preference Pane" width="300" height="292" /></a><p class="wp-caption-text">Parental Controls Preference Pane (Click to Enlarge)</p></div>
<p>Create an admin account on the Mac Mini.  While in this account, create a new user (“Producer”), and enable Parental Controls on the account.  In Parental Controls on the Producer account, activate “Use Simple Finder” and “Only allow selected applications”.  In this box, select the application Front Row.  Make other changes at your discretion, but these are the biggies.  This will ensure that this user has limited access to OS functions and can only run Front Row.</p>
<p>Now, we need a way to have the Mac Mini see the shared / network drive without user intervention.  Automator is an easy way to accomplish this.</p>
<p>Open Automator.  Create a workflow that:</p>
<p>Gets Specified Servers and make sure to put the username and password in the URL; i.e.: afp://username:password@192.168.1.1/DRIVE_FOLDER_WITH_ENCODED_MEDIA</p>
<p>Add a “Connect to Servers” Action.</p>
<p>You can download the workflow here: <a href="http://michaelkammes.com/wp-content/uploads/2009/05/producer_automount.zip">producer_automount</a>.</p>
<div id="attachment_223" class="wp-caption alignright" style="width: 310px"><a href="http://michaelkammes.com/wp-content/uploads/2009/05/automator.gif"><img class="size-medium wp-image-223  " title="Automator Network Share Mount Workflow" src="http://michaelkammes.com/wp-content/uploads/2009/05/automator-300x191.gif" alt="Automator Network Share Mount Workflow" width="300" height="191" /></a><p class="wp-caption-text">Automator Network Share Mount Workflow (Click to Enlarge)</p></div>
<p>Save this as an Application.</p>
<p>Run this to test.  Does it work?  Great!</p>
<p>Now, create an Alias to this shared / network drive.  Place this alias in the Movies folder on your Producer account.  This allows Front Row to see movies located on the shared / network volume.  You can change the name of the alias if you wish to something a bit more verbose.</p>
<p>Place the Automator Application you’ve created in the Login Items under the Producer account.  Also, add Front Row.  Set the system to auto boot into the Producer account.  In System Preferences, disable any Bluetooth keyboard / mouse options – so the Mac doesn’t look for Bluetooth devices.  If this is not disabled, the Mac will look for devices on bootup, potentially stalling the launching of the Automator Application and Front Row.  Reboot.  As the system is rebooting, disconnect your keyboard, mouse, and make sure you have your Apple Remote handy.</p>
<p>When the computer reboots, it will boot into the Producer account, auto mount the network volume which has the media you want them to see, then launches Front Row.</p>
<p>Now, using the Apple Remote, navigate through Front Row, and through the MOVIES section.  Because we placed an alias to the network drive (which the computer is connected to via our Automator script), Front Row can now see that drive.</p>
<p>Success!</p>
<p>Now, you have the ability to play back ANY file format QT can play back, from an unlimited number of machines AND restrict access to what clips can be seen by mounting different network shares though Automator.</p>
<p>As an added bonus, if the Producer decides to think outside the proverbial box, and hit the MENU key on the remote while at the Front Row main menu – it will take them to the OS desktop.  Surprise!  No keyboard and no mouse make tampering very difficult, PLUS you’ve already locked out OS functions with Parental Controls.  All they can do is turn the Mac Mini volume up and down…. and simply hitting the MENU button relaunches Front Row.</p>
<p>Success! Easy edit review with a nice Apple sheen.</p>
]]></content:encoded>
			<wfw:commentRss>http://michaelkammes.com/editorial/a-little-somethin-for-the-producers-mass-review-and-approval-made-simple/feed/</wfw:commentRss>
		<slash:comments>5</slash:comments>
		</item>
		<item>
		<title>Post NAB Rumors</title>
		<link>http://michaelkammes.com/avid/post-nab-rumors/</link>
		<comments>http://michaelkammes.com/avid/post-nab-rumors/#comments</comments>
		<pubDate>Wed, 22 Apr 2009 21:47:57 +0000</pubDate>
		<dc:creator>Michael Kammes</dc:creator>
				<category><![CDATA[Avid]]></category>
		<category><![CDATA[Storage]]></category>
		<category><![CDATA[Apple]]></category>
		<category><![CDATA[Final Cut Pro]]></category>
		<category><![CDATA[NAB]]></category>

		<guid isPermaLink="false">http://michaelkammes.com/column/?p=205</guid>
		<description><![CDATA[I know it's customary to hear rumors prior to a show, not after...but it's quite interesting when the brain trusts of the industry get together and swap stories.]]></description>
			<content:encoded><![CDATA[<p>I know it&#8217;s customary to hear rumors <em>prior</em> to a show, not after&#8230;but it&#8217;s quite interesting when the brain trusts of the industry get together and swap stories.</p>
<p>Among the goodies:</p>
<p>Final Cut Studio 3:  Increased RED support <em>(confirmed)</em>, as well as <strong>no</strong> support for blu-ray within DVD Studio Pro.  I am unsure if this means no authoring ability or the user not being able to even build a project.</p>
<p>Avid:  Specs for Nehalem support.  This is, of course, unofficial.</p>
<p>Nehalem Mac Pro</p>
<p>OS 10.5.6</p>
<p>2.66 or 2.93 Quad</p>
<p>Core Intel Xeon processors (8-core)</p>
<p>6BG of 1066 DDR3 ECC SDRAM</p>
<p>NVIDIA GeForce GT 120 512MB**</p>
<p>640GB 7200-rpm SATA 3Gb/s</p>
<p>**It should be mentioned that I did not see any Mini Display Port extenders on the NAB show floor, which make any Nehalem Mac install extraordinarily difficult.  One can, of course, convert to DVI, as a fallback.</p>
<p>More as they trickle in.</p>
]]></content:encoded>
			<wfw:commentRss>http://michaelkammes.com/avid/post-nab-rumors/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Cheatsheet for Stereoscopic in Avid Media Composer 3.5</title>
		<link>http://michaelkammes.com/editorial/cheatsheet-for-stereoscopic-in-avid-media-composer-35/</link>
		<comments>http://michaelkammes.com/editorial/cheatsheet-for-stereoscopic-in-avid-media-composer-35/#comments</comments>
		<pubDate>Wed, 11 Mar 2009 02:55:34 +0000</pubDate>
		<dc:creator>Michael Kammes</dc:creator>
				<category><![CDATA[Avid]]></category>
		<category><![CDATA[Editorial]]></category>
		<category><![CDATA[Storage]]></category>

		<guid isPermaLink="false">http://michaelkammes.com/column/?p=17</guid>
		<description><![CDATA[Summary: Avid Media Composer now has the ability to edit stereoscopic footage, by combining the 2 separate “eyes” into a single video stream. Launch Metafuze on a PC. (comes with the Avid installer, or download here.) Import the Left Eye &#38; Right Eye (in that order) Folders. Avid calls this scanning. These will have unique file names, but the numeric values in each file name will match Left Eye &#38; Right Eye files, and must have the same file format (size &#38; resolution) and same duration. Highlight a Left Eye file AND a Right Eye file. On the right window ...]]></description>
			<content:encoded><![CDATA[<p>Summary:  Avid Media Composer now has the ability to edit stereoscopic footage, by combining the 2 separate “eyes” into a single video stream.</p>
<ul>
<li>
<div>Launch Metafuze on a PC. (comes with the Avid installer, or download <a href="http://www.avid.com/metafuze/" target="_blank">here</a>.)</div>
</li>
<li>
<div>Import the Left Eye &amp; Right Eye (in that order) Folders. Avid calls this scanning. These will have unique file names, but the numeric values in each file name will match Left Eye &amp; Right Eye files, and must have the same file format (size &amp; resolution) and same duration.</div>
</li>
<li>
<div>Highlight a Left Eye file AND a Right Eye file.  On the right window pane there are transcoding options.  Select your <a href="http://www.avid.com/dnxhd/" target="_blank">flavor of DNxHD</a>, and Click the Create Stereo Group button.</div>
</li>
<li>
<div>You now have Avid appropriate (OP1a) MXF wrapped video files which can be loaded into Avid (without Avid having to re-wrap the file).  Utilizing the Stereoscopic settings in Avid, you can now view Left Eye or Right Eye only, over/under, or checkerboard.  Remember, you can only view your 3D playout correctly via the DVI port on your computer – not via the baseband video output on your Mojo or Nitris DX.  Don’t forget your IR 3D glasses and 3D capable display!</div>
</li>
</ul>
]]></content:encoded>
			<wfw:commentRss>http://michaelkammes.com/editorial/cheatsheet-for-stereoscopic-in-avid-media-composer-35/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Tech Note: Avid and Volume Sizes</title>
		<link>http://michaelkammes.com/editorial/tech-note-avid-and-volume-sizes/</link>
		<comments>http://michaelkammes.com/editorial/tech-note-avid-and-volume-sizes/#comments</comments>
		<pubDate>Fri, 09 Jan 2009 02:40:58 +0000</pubDate>
		<dc:creator>Michael Kammes</dc:creator>
				<category><![CDATA[Avid]]></category>
		<category><![CDATA[Editorial]]></category>
		<category><![CDATA[Finishing]]></category>
		<category><![CDATA[Storage]]></category>
		<category><![CDATA[Tech Note]]></category>
		<category><![CDATA[Windows]]></category>

		<guid isPermaLink="false">http://michaelkammes.com/column/?p=12</guid>
		<description><![CDATA[Best Practices - Drive partitioning suggestions from Avid Customer Support.]]></description>
			<content:encoded><![CDATA[<p><span style="color: #888888;"><span style="font-family: Arial,Helvetica; font-size: x-small;">Current harddrives and storage solutions are available containing terrabytes of space. While the computer itself can support that much space, after 250+GB of storage has been taken up with media, the Avid begins to lose its way when dealing with Media DataBases. This can lead to the system &#8216;losing&#8217; media or incorrectly show media as &#8220;Offline&#8221;. Also, very often the Media Databases may not be successfully rebuilding themselves (the <strong>msm.MMOB</strong> and <strong>msm.FMID</strong> files located inside each of the <strong>OMFI MediaFiles Folder</strong>, which is causing speed and stability problems as users begin to fill up the drives with media.</span></span></p>
<p><span style="color: #888888;"><span style="font-family: Arial,Helvetica; font-size: x-small;"><strong>Here is what is going on with your media, as you work within the Avid:</strong></span></span></p>
<ul>
<li><span style="color: #888888;"><span style="font-family: Arial,Helvetica; font-size: x-small;">Almost everything you are doing in Avid (from launching the Avid, saving any bins, leaving capture mode, or closing the application ) is causing the system to read and write data to your media drives simultaneously.</span></span></li>
<li><span style="color: #888888;"><span style="font-family: Arial,Helvetica; font-size: x-small;">Each partition will have one <strong>OMFI MediaFiles Folder</strong> and/or one <strong>Avid Media Files Folder</strong> (for MXF media).</span></span></li>
<li><span style="color: #888888;"><span style="font-family: Arial,Helvetica; font-size: x-small;">Each media files folder contains two database files (<strong>msm.MMOB</strong> and <strong>msm.FMID</strong>) which keep track of <span style="text-decoration: underline;">all</span> of the changes to media files; keeping them constantly linked to the master clips in your bins (pointer files living on another drive in your applications folder).</span></span></li>
<li><span style="color: #888888;"><span style="font-family: Arial,Helvetica; font-size: x-small;">Every time you launch the Avid, save any bins, leave capture mode, or close the application it is forcing those database files to index all of the files in that media files folder.</span></span></li>
<li><span style="color: #888888;"><span style="font-family: Arial,Helvetica; font-size: x-small;">By having 2 database files on one very large partition containing 500GB 1 TB, etc of media , the Avid is indexing the hundreds of thousands of files included on that partition . That, in turn, taxes the database files beyond a reasonable level and they begin to either lose track of media, or fail to complete their updates.</span></span></li>
<li><span style="color: #888888;"><span style="font-family: Arial,Helvetica; font-size: x-small;">Media Databases no longer update correctly, causing things like &#8220;OMFI HP Domain INIT Failed&#8221; errors as well as &#8216;Media OffLine&#8221; messages, among others.</span></span></li>
<li><span style="color: #888888;"><span style="font-family: Arial,Helvetica; font-size: x-small;">By creating more, yet smaller Drive Partitions, the Avid can more efficiently and reliably maintain its Media Database files, giving overall stability and speed back to the system.</span></span></li>
</ul>
<p><span style="color: #888888;"><span style="font-family: Arial,Helvetica; font-size: x-small;">Avid Customer Support has found that partitioning down larger striped sets to sizes of approximately 250-300 GB has TREMENDOUSLY helped overall performance and stability. We recommend this for systems setup with larger than 300 GB partitions, in use with a 12 or 24 way stripe.</span></span></p>
<p><span style="color: #888888;"><span style="font-family: Arial,Helvetica; font-size: x-small;"><strong>A note about Windows File Limit:</strong></span></span></p>
<p><span style="color: #888888;"><span style="font-family: Arial,Helvetica; font-size: x-small;">The Windows OS starts to slow down after 10,000 files. This is a limitation of the OS itself, not of the Avid. After 10,000 files, the system cannot buffer it&#8217;s file count fast enough, as well as the Avid&#8217;s Media Databases also not updating correctly, if at all.</span></span></p>
<p><span style="color: #888888;"><span style="font-family: Arial,Helvetica; font-size: x-small;">More info available at:  <a href="http://avidtechnology.custhelp.com/cgi-bin/avidtechnology.cfg/php/enduser/std_adp.php?p_faqid=71457">http://avidtechnology.custhelp.com/cgi-bin/avidtechnology.cfg/php/enduser/std_adp.php?p_faqid=71457</a><br />
</span></span></p>
]]></content:encoded>
			<wfw:commentRss>http://michaelkammes.com/editorial/tech-note-avid-and-volume-sizes/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
	</channel>
</rss>
<!-- This Quick Cache file was built for (  www.michaelkammes.com/category/storage/feed/ ) in 2.26899 seconds, on Feb 4th, 2012 at 5:45 am UTC. -->
<!-- This Quick Cache file will automatically expire ( and be re-built automatically ) on Feb 4th, 2012 at 6:45 am UTC -->
