<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Gamze's Data Notes]]></title><description><![CDATA[Gamze's Data Notes]]></description><link>https://gamzenuraslan.hashnode.dev</link><image><url>https://cdn.hashnode.com/res/hashnode/image/upload/v1593680282896/kNC7E8IR4.png</url><title>Gamze&apos;s Data Notes</title><link>https://gamzenuraslan.hashnode.dev</link></image><generator>RSS for Node</generator><lastBuildDate>Wed, 16 Sep 2026 17:33:48 GMT</lastBuildDate><atom:link href="https://gamzenuraslan.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[I Finally Understood Why We Need OLTP, OLAP, Data Lakes and Lakehouses]]></title><description><![CDATA[Lately I have been spending more time on data engineering and one question kept coming back to me
Where should the data actually live
At first this sounded like a simple storage decision. Then I reali]]></description><link>https://gamzenuraslan.hashnode.dev/i-finally-understood-why-we-need-oltp-olap-data-lakes-and-lakehouses</link><guid isPermaLink="true">https://gamzenuraslan.hashnode.dev/i-finally-understood-why-we-need-oltp-olap-data-lakes-and-lakehouses</guid><category><![CDATA[data-engineering]]></category><category><![CDATA[Data warehouse]]></category><category><![CDATA[Data-lake]]></category><category><![CDATA[big data]]></category><dc:creator><![CDATA[aslangamzenur079]]></dc:creator><pubDate>Fri, 11 Sep 2026 19:57:47 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6aa4559b22cd40cbd46735f9/dd537dfa-734d-4685-b8f2-a75234ab687b.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Lately I have been spending more time on data engineering and one question kept coming back to me</p>
<p>Where should the data actually live</p>
<p>At first this sounded like a simple storage decision. Then I realized that where we keep data can affect performance, cost and even how easy it is to work with that data later</p>
<p>The example that made everything click for me was surprisingly simple</p>
<p>Imagine an application that handles thousands of orders every day. Users are creating orders, updating them and checking their status</p>
<p>Now imagine running a huge analytical query on the same database while all of this is happening</p>
<p>The query may work</p>
<p>But the system can slow down because the database is now trying to serve two very different workloads at the same time</p>
<p>That is where OLTP and OLAP started to make much more sense to me</p>
<h2>OLTP and OLAP are built for different jobs</h2>
<p>OLTP systems are designed for everyday operational work</p>
<p>Creating an order<br />Updating a shipment<br />Changing a customer record</p>
<p>These are usually small and fast operations</p>
<p>OLAP systems are designed for analysis</p>
<p>How many orders did we receive last month<br />Which city had the highest delivery volume<br />What was the average shipment weight this year</p>
<p>These questions may require scanning a lot more data</p>
<p>So instead of forcing one system to do everything, operational and analytical workloads are often separated</p>
<p>That small idea helped me understand why data warehouses exist in the first place</p>
<h2>Then I started looking at Data Lakes</h2>
<p>A Data Lake gives us much more freedom</p>
<p>Instead of putting everything into a strict table structure immediately, we can store raw files, logs, JSON data and many other formats and decide how to process them later</p>
<p>This makes Data Lakes flexible and often cheaper for large amounts of data</p>
<p>But flexibility also creates another problem</p>
<p>If the data is not managed well, it can become difficult to understand what is reliable and what is not</p>
<p>That is where the Lakehouse idea became interesting to me</p>
<h2>What made Lakehouse interesting</h2>
<p>Lakehouse tries to combine the flexibility of a Data Lake with some of the reliability and structure we expect from a Data Warehouse</p>
<p>Technologies such as Delta Lake and Apache Iceberg add features like table history, schema management and transaction support on top of file based storage</p>
<p>So instead of thinking only in terms of Data Lake versus Data Warehouse, there is another approach that tries to bring both sides together</p>
<p>I found this useful because it made the whole storage discussion feel less like choosing one perfect system and more like understanding the tradeoffs</p>
<h2>Partitioning was another small concept that made a big difference</h2>
<p>Imagine having five years of shipment data</p>
<p>If we only want to analyze yesterday's shipments, reading all five years would be unnecessary</p>
<p>If the data is partitioned by date, the system can focus only on the part we actually need</p>
<p>That means less data scanned and usually faster queries</p>
<p>In cloud environments it can also mean lower cost because many services charge based on how much data is processed</p>
<p>This was one of the parts that surprised me most</p>
<p>Something that looks like a simple storage decision can directly affect both performance and cost</p>
<h2>What I took away from this</h2>
<p>The more I learn about data engineering, the more I see that storage is not just about finding a place to put data</p>
<p>The real question is what kind of work we expect that data to support</p>
<p>Operational systems need speed for small transactions</p>
<p>Analytical systems need efficient scans over large amounts of data</p>
<p>Data Lakes give us flexibility</p>
<p>Lakehouse architectures try to add more structure and reliability</p>
<p>And choices like partitioning can change how much work the system needs to do</p>
<p>I am still learning where the line between these approaches should be drawn in real projects</p>
<p>If you work with data, I would genuinely like to know</p>
<p>Which one do you see more often in practice today</p>
<p>Data Warehouse, Data Lake or Lakehouse</p>
]]></content:encoded></item></channel></rss>