Mada za sehemu hiiCollecting and storing dataMada 4
- Collecting data from multiple sources
- Implementing retrieval of data from files
- Implementing local data storage
- Performing connection to remote databases
When we process data in Python, we often need to keep a copy of the results on our computer for later use. This is called local data storage — saving processed data to files on your hard drive or removable storage so you can open and use them again without re-running your script.
This topic shows you how to write data from Python programs into two common flat-file formats: CSV (Comma-Separated Values) and JSON (JavaScript Object Notation). Both formats are plain text files that can be opened in many applications, including spreadsheets and text editors.
A CSV file stores data in rows, with each value separated by a comma. It looks like a simple table and is widely supported.
Using pandas to Write CSV
The pandas library provides a simple function called to_csv() to save a DataFrame to a CSV file.
General syntax:
dataframe.to_csv('filename.csv', index=False)
- The first argument is the name of the file you want to create.
- The parameter
index=Falsetells Python not to write the row numbers into the file.
Worked Example
Imagine a small shop in Arusha recorded the sales of three items:
| Item | Quantity | Unit Price (TSh) |
|---|---|---|
| Rice | 50 | 1500 |
| Sugar | 30 | 2000 |
| Flour | 20 | 1800 |
The following Python script saves this data to a file called sales_data.csv:
import pandas as pd
# Create the data
data = {
'Item': ['Rice', 'Sugar', 'Flour'],
'Quantity': [50, 30, 20],
'Unit Price (TSh)': [1500, 2000, 1800]
}
# Make a DataFrame
df = pd.DataFrame(data)
# Save to CSV
df.to_csv('sales_data.csv', index=False)
print("Data saved to sales_data.csv successfully!")
When you open sales_data.csv in a text editor, it looks like:
Item,Quantity,Unit Price (TSh)
Rice,50,1500
Sugar,30,2000
Flour,20,1800
JSON stores data in a format that looks like Python dictionaries. It is especially useful when your data has a hierarchical structure (nested data).
Using the json Library
The standard json library can write Python dictionaries to JSON files.
General syntax:
import json
with open('filename.json', 'w') as file:
json.dump(data_dictionary, file, indent=number)
'w'opens the file for writing.indentformats the file so each level is easier to read.
Worked Example
Using the same shop data but as a list of dictionaries:
import json
# Create the data as a list of dictionaries
sales_data = [
{"Item": "Rice", "Quantity": 50, "Unit Price (TSh)": 1500},
{"Item": "Sugar", "Quantity": 30, "Unit Price (TSh)": 2000},
{"Item": "Flour", "Quantity": 20, "Unit Price (TSh)": 1800}
]
# Save to JSON
with open('sales_data.json', 'w') as file:
json.dump(sales_data, file, indent=4)
print("Data saved to sales_data.json successfully!")
The file sales_data.json contains:
[
{
"Item": "Rice",
"Quantity": 50,
"Unit Price (TSh)": 1500
},
{
"Item": "Sugar",
"Quantity": 30,
"Unit Price (TSh)": 2000
},
{
"Item": "Flour",
"Quantity": 20,
"Unit Price (TSh)": 1800
}
]
When choosing how to store data locally, consider what you need:
| Feature | CSV | JSON | Excel |
|---|---|---|---|
| File size | Very small | Small to medium | Larger |
| Human readable | Yes (simple tables) | Yes (structured text) | Yes (tables) |
| Supports complex data | No (flat tables only) | Yes (nested structures) | Yes |
| Editing in spreadsheets | Yes | No (needs special tools) | Yes |
| Speed with large data | Fast | Fast | Slower |
- Use CSV when you have simple tabular data and want fast, lightweight storage that works with almost any software.
- Use JSON when your data includes lists inside lists or when you need to exchange data with web applications.
- Use Excel (.xlsx) when you need multiple sheets, rich formatting, or when non-programmers will edit the file.
A student running a small poultry farm in Dodoma can use Python to record daily egg production in a CSV file. By writing a short script that saves the date, number of eggs collected, and price per egg (in TSh) to egg_sales.csv, the farmer keeps a permanent backup of sales records. Later, they can load this file into a spreadsheet to calculate monthly income or track trends — all without re-entering data by hand.
Jinsi utakavyotathminiwaTathmini ya mchakato · Tathmini ya matokeo · Tathmini ya maarifa
Tathmini ya mchakato
- Write processed data using appropriate file formats.
- Proper naming and directory organization for stored files.
Tathmini ya matokeo
A well-structured local directory with saved datasets in appropriate formats.
Tathmini ya maarifa
Detailed knowledge of:
Method used: The student should explain how to:
- save cleaned and formatted files to a local computer.
Principles: The student should explain the principles of
- file format selection for data analysis and sharing.
Theories: The student should explain:
- Concepts of file formats.
Circumstantial knowledge:
Detailed knowledge about:
- Handling large datasets that might exceed memory limits during saving.
Swali
What does the parameter index=False do when saving a DataFrame to a CSV file using pandas?
Ingia ili kuwasilisha jibu lako na lihesabiwe katika umahiri wako.
Ingia ili kufanya mazoeziMwalimu
Umekwama? Niulize chochote kuhusu mada hii.
Ingia ili kumuuliza Mwalimu wa AI wa Sonza kuhusu swali hili.
Ingia ili kuulizaMajadiliano
Hakuna maswali bado