Report logo

The State of Data 2024

782 responses

What is your current role or position in your organization?

782 out of 782 answered

Senior
315 resp.
40.3%
Mid-level
232 resp.
29.7%
Entry-level
87 resp.
11.1%
Manager
67 resp.
8.6%
C-Suite
42 resp.
5.4%
VP
18 resp.
2.3%
Other
21 resp.
2.7%
Solo
Freelancer
Freelance
Owner
my own company
All the things
Chief Solutions Architect
Owner
Freelancer
Freelance Web Developer (10+ years)
Student
Student
Founder
Principal
student
Consultant
Junior
Owner
Unemployed
Indie developer
startup

Which department or team do you primarily work in?

782 out of 782 answered

Engineering
484 resp.
61.9%
IT
173 resp.
22.1%
Product
58 resp.
7.4%
Data Science
22 resp.
2.8%
Operations
18 resp.
2.3%
Marketing/Sales
12 resp.
1.5%
Human Resources
0 resp.
0%
Other
15 resp.
1.9%
Web developer
All the things
Design
Customer support
Communications
Engineering, operations, product
Teaching
Development
Media and Film
Data Analytics
Consultant
Creative
Front and developer
Data
Everything

Which industry best represents your organization's primary focus?

782 out of 782 answered

Technology
452 resp.
57.8%
Retail
63 resp.
8.1%
Finance
56 resp.
7.2%
Healthcare
40 resp.
5.1%
Education
35 resp.
4.5%
Manufacturing
28 resp.
3.6%
Government
23 resp.
2.9%
Non-Profit
13 resp.
1.7%
Other
72 resp.
9.2%
Any type of industry
Media
Recruitment
Marketing
Assurance
Energy
Electricity
Mobility
real estate
Social / Gaming
Marketing
Agency
Media
Real Estate
Agency
Innovation
real-estate
Sport
Transport
Agriculture
Any, usually marketing sites for tech, restaurants, music, etc.
Property
Service & installation
Gaming
Web content

How big is your organization in terms of employees?

782 out of 782 answered

10-49
187 resp.
23.9%
1-9
162 resp.
20.7%
100-499
130 resp.
16.6%
5000+
100 resp.
12.8%
50-99
92 resp.
11.8%
1000-4999
57 resp.
7.3%
500-999
54 resp.
6.9%

How many years have you been working with data professionally?

782 out of 782 answered

1-3 years
230 resp.
29.4%
More than 10 years
185 resp.
23.7%
4-6 years
173 resp.
22.1%
7-10 years
117 resp.
15%
Less than 1 year
77 resp.
9.8%

Which of the following activities are you most involved with in your role?

782 out of 782 answered

Fullstack development/management
514 resp.
65.7%
Frontend development/management
380 resp.
48.6%
Technical leadership
252 resp.
32.2%
Backend development/management
207 resp.
26.5%
Project management
206 resp.
26.3%
Data analysis
105 resp.
13.4%
Security/privacy
105 resp.
13.4%
Data engineering
96 resp.
12.3%
AI/ML
50 resp.
6.4%
Other
15 resp.
1.9%
DevOps
Products and Solutions
Embedded development
Growth marketing, SEO, ads
Front & back end dev
DevOps
software help
IT Operations
QA
DevOps
UX/UI
Mobile development
Lead
Sales / Marketing
Robotic Process Automation

Which region are you based in?

782 out of 782 answered

Europe
378 resp.
48.3%
North America
209 resp.
26.7%
Asia
104 resp.
13.3%
Oceania
34 resp.
4.3%
South America
34 resp.
4.3%
Africa
17 resp.
2.2%
Prefer not to say
6 resp.
0.8%

What database(s) are you currently using in your projects?

782 out of 782 answered

PostgreSQL
499 resp.
63.8%
MySQL
370 resp.
47.3%
MongoDB
265 resp.
33.9%
Redis
254 resp.
32.5%
SQLite
185 resp.
23.7%
Microsoft SQL Server
131 resp.
16.8%
MariaDB
118 resp.
15.1%
Elasticsearch
90 resp.
11.5%
Amazon DynamoDB
80 resp.
10.2%
Oracle Database
53 resp.
6.8%
Cloudflare D1
28 resp.
3.6%
Neo4j
27 resp.
3.5%
Turso
17 resp.
2.2%
Pinecone
13 resp.
1.7%
CouchDB
9 resp.
1.2%
Chroma
8 resp.
1%
SAP HANA
8 resp.
1%
CockroachDB
7 resp.
0.9%
Apache Cassandra
6 resp.
0.8%
Weaviate
6 resp.
0.8%
IBM Db2
5 resp.
0.6%
Couchbase
4 resp.
0.5%
FaunaDB
3 resp.
0.4%
SingleStore
1 resp.
0.1%
HarperDB
0 resp.
0%
Milvus
0 resp.
0%
Other
59 resp.
7.5%
Supabase
Clickhouse
Firebase
Hacky Shit
Firestore
VistA
Firestore
RavenDB
axiom.co
Firebase
GROQ (schemaless)
ClickHouse
Google Data Store
Snowflake
Firestore
Airtable
SurrealDB
XTDB
Firestore
Google Firestore and Google Spanner
Qdrant
Duckdb
Solr
influxdb
Clickhouse, Duckdb, HBase
Aurora
DynamoDB
Firestore
Snowflake
Webflow CMS
Algolia
optimizely
Firestore
BigQuery
None
Firestore
Clickhouse
Airtable
AuroraDB
TimescaleDB
Firebase
CosmosDB
Microsoft Visual Foxpro
CosmosDB
GCP
Progress
DuckDB
Airtable, Convex
Firestore
Not working with DB

Do you have a general preference for a database type in new projects?

782 out of 782 answered

Relational databases
449 resp.
57.4%
Depends on the project
138 resp.
17.6%
NoSQL databases
97 resp.
12.4%
No strong preference
79 resp.
10.1%
Graph databases
9 resp.
1.2%
Time-series databases
4 resp.
0.5%
Search databases
2 resp.
0.3%
Vector databases
2 resp.
0.3%
NewSQL databases
0 resp.
0%
Other
2 resp.
0.3%
N/A
i do frontend

Where do you currently host your databases?

782 out of 782 answered

Cloud service providers (e.g. AWS, Azure, GCP)
533 resp.
68.2%
Managed database services (e.g. Amazon RDS, Google Cloud SQL, Digital Ocean)
260 resp.
33.2%
On-premises servers
196 resp.
25.1%
Database-as-a-Service providers (e.g. PlanetScale, FaunaDB, Xata)
119 resp.
15.2%
Shared hosting (e.g. Dreamhost, GoDaddy, IONOS)
60 resp.
7.7%
Dedicated hosting providers (e.g. IBM Cloud, Rackspace, Liquid Web)
53 resp.
6.8%
Other
25 resp.
3.2%
Currently using an electron app locally
Managed Servers (Mittwald)
Vps
On server VPS
Railway
Managed Server (Hetzner)
Self hosted on a VPS.
Railway, Render, Flyio
Docker
Railway
Other company
Fly.io
Platform-as-a-Service (similar to Directus Cloud)
On demand
Localhost
Mongodb atlas
Can not answer
Nowhere
I don't know how you call it, it's clevercloud
Shared Hosting
Vendor hosted, aka Airtable
N/A
Self hosting
upstash
Other

How do you consider cost when choosing a hosting solution for databases?

782 out of 782 answered

Cost is a consideration, but not the primary factor
357 resp.
45.7%
I don't handle cost considerations
192 resp.
24.6%
Cost is the primary factor
186 resp.
23.8%
Cost is not a significant concern
47 resp.
6%
Other
0 resp.
0%

What factors do you think should primarily influence the cost of hosting?

782 out of 782 answered

Machine resource utilization (e.g. RAM/CPU)
469 resp.
60%
Storage space
402 resp.
51.4%
Number of read/write operations
332 resp.
42.5%
Bandwidth usage
313 resp.
40%
Number of concurrent connected users
240 resp.
30.7%
Number of database records
114 resp.
14.6%

How do you currently ensure the scalability of your data infrastructure?

782 out of 782 answered

Horizontal scaling (adding more machines)
324 resp.
41.4%
Load balancing
311 resp.
39.8%
Vertical scaling (upgrading existing machines)
271 resp.
34.7%
My hosted service manages scaling
231 resp.
29.5%
No specific measures for scalability
124 resp.
15.9%
Sharding the database
108 resp.
13.8%
Other
8 resp.
1%
We didn't need to scale yet
We deploy more pods in k8 if needed
One database per user (Turso)
Not in my power, some other team takes care of that
?
Idk
N/A
All of the above

What is your preferred practice for handling demand spikes?

782 out of 782 answered

Automatically scale resources with cap
230 resp.
29.4%
Anticipate demand, temporarily increase resources
156 resp.
19.9%
Over-provision, always have available resources
109 resp.
13.9%
Outsource to a hosting provider
105 resp.
13.4%
Reactively, manually, increase resources
95 resp.
12.1%
Implement load-shedding (throttle, delay, or reject requests)
35 resp.
4.5%
Automatically scale resources without cap
34 resp.
4.3%
Other
18 resp.
2.3%
Application doesn't have big spikes, not need for now
I don't handle cloud deployments
I have no involvement with scaling
It's never been a problem
Won’t happen anyway
Airtable doesn't allow for that kind of control
I don't have those yet...
idk
I don't deal with this
Runs serverless
not handled
N/A
Don't scale up 🙈
We know the maximum possible peaks based on the periodof the year
We have defined user caps.
Idk
Yet to handle any spikes.
N/A

Where do you implement data caching in your applications?

782 out of 782 answered

Client-side caching
459 resp.
58.7%
Server in-memory caching
353 resp.
45.1%
Database-level caching
208 resp.
26.6%
Distributed/edge caching
200 resp.
25.6%
Dedicated caching proxy server
143 resp.
18.3%
No specific caching strategies
119 resp.
15.2%
Other
9 resp.
1.2%
Server file caching
Redis
Redis query->response cache
Application in process caching
N/A
Materialized views, varnish
Most data is stored in Camunda processes
N/A
Web server handles caching data

Have you ever made efforts to reduce response times through distributed compute?

782 out of 782 answered

No, I am not interested or it's not relevant to my projects
237 resp.
30.3%
Yes, my applications are partially distributed
162 resp.
20.7%
No, but I am assessing the feasibility of distributed compute
153 resp.
19.6%
No, there are limitations in our current infrastructure or application
122 resp.
15.6%
Yes, my applications are fully distributed
108 resp.
13.8%

What encryption practices do you implement to secure data in transit?

782 out of 782 answered

TLS/SSL (HTTPS)
720 resp.
92.1%
VPNs (Virtual Private Networks)
315 resp.
40.3%
Signing with PGP Keys
94 resp.
12%
IPsec
61 resp.
7.8%
No specific encryption practices
35 resp.
4.5%
Other
13 resp.
1.7%
localy hosted on-premise
access lists
Custom encryption layer to ensure data safety
idk
MFA
I don't know in the current project
JWT
Can't disclose
Trust and watermarks
WSS
N/A
VPC
Physical card sign in

Have you ever implemented the following access control measures within your applications?

782 out of 782 answered

Role-based access control (RBAC)
597 resp.
76.3%
Multi-factor authentication (MFA)
431 resp.
55.1%
Attribute-based access control (ABAC)
131 resp.
16.8%
Regular access reviews
125 resp.
16%
No specific access control measures
97 resp.
12.4%

Which of the following data processing practices do you actively work to comply with?

782 out of 782 answered

GDPR (General Data Protection Regulation) compliance
512 resp.
65.5%
Other regional or industry-specific regulations
205 resp.
26.2%
No specific compliance measures
166 resp.
21.2%
HIPAA (Health Insurance Portability and Accountability Act) compliance
104 resp.
13.3%
CCPA (California Consumer Privacy Act) compliance
91 resp.
11.6%
Other
8 resp.
1%
Lei Geral de Proteção de Dados (Brazil)
LGPD (Brazil's version of data privacy laws)
SOC II
idk
SOC
Idk
SOC2
PSD2

When and where does your app load data?

782 out of 782 answered

During use (async)
663 resp.
84.8%
On first load (hydrate)
407 resp.
52%
On build (static)
302 resp.
38.6%
On navigation (navigation guard)
230 resp.
29.4%

How do you handle data streaming or event-driven architectures in your applications?

782 out of 782 answered

WebSocket protocols
346 resp.
44.2%
Polling for updates
270 resp.
34.5%
No specific streaming methods
240 resp.
30.7%
Apache Kafka or similar messaging systems
144 resp.
18.4%
Server-Sent Events (SSE)
137 resp.
17.5%
Other
10 resp.
1.3%
Firebase
SignalR
Firebase
Background azure functions
Sidekiq
I don't know in my current project
Subscriptions
I dont know
MQTT
N/A

Do you use islands architecture / micro-frontends?

782 out of 782 answered

Heard of it and would like to learn more
285 resp.
36.4%
Used it and would use again
175 resp.
22.4%
Never heard of it / don't know what it is
167 resp.
21.4%
Heard of it and not interested
122 resp.
15.6%
Used it and would not use again
33 resp.
4.2%

Do you use global store?
(Vuex / Pinia / Context / Redux / Flux)

782 out of 782 answered

Used it and would use again
455 resp.
58.2%
Used it and would not use again
103 resp.
13.2%
Heard of it and would like to learn more
95 resp.
12.1%
Heard of it and not interested
68 resp.
8.7%
Never heard of it / don't know what it is
61 resp.
7.8%

Do you use composables / hooks?

782 out of 782 answered

Used it and would use again
572 resp.
73.1%
Heard of it and would like to learn more
94 resp.
12%
Never heard of it / don't know what it is
75 resp.
9.6%
Heard of it and not interested
22 resp.
2.8%
Used it and would not use again
19 resp.
2.4%

When working with APIs, what interface do you prefer?

782 out of 782 answered

REST
733 resp.
93.7%
GraphQL
332 resp.
42.5%
WebSocket
220 resp.
28.1%
gRPC
95 resp.
12.1%
MQTT
43 resp.
5.5%
Other
11 resp.
1.4%
tRPC
TRPC
HTTP not tied to REST
tRPC
tRPC
oData
TSRPC
notion
tRPC
Endpoints that do what you want them to do (so not REST)
tRPC & RSA

Which authentication methods do you commonly use when working with APIs?

782 out of 782 answered

JWT (JSON Web Tokens)
577 resp.
73.8%
API keys
559 resp.
71.5%
OAuth
432 resp.
55.2%
Basic Auth (username/password)
249 resp.
31.8%
Other
13 resp.
1.7%
Session tokens
maintaining session
JWT's are API Keys in the context of authenticating to APIs
Bearer
keycloak
.NET identity
cookie
NTLM
Cookies
Magic Link (Email)
N/A
sessions
LTI OIDC Login with LTI Client Side postMessages

What are your biggest challenges in using APIs?

782 out of 782 answered

Inconsistent data formats
389 resp.
49.7%
Handling errors and exceptions
347 resp.
44.4%
Understanding API documentation
305 resp.
39%
Integrating APIs with existing systems
266 resp.
34%
Managing API keys and access tokens
227 resp.
29%
Securing API calls
227 resp.
29%
Other
16 resp.
2%
G. Lack hof programming knowledge
Return schemas
Network security access issues.
Missing documentation
Lack of example code
data volume
API usage is not a challenge.
Missing, incomplete & inaccurate API documentation
throttling
no challenges stick out
Checking if extending API should be part of current version with backward compatibility or create new version for the change
None
Managing redirects
There's not enough time in the world to implement, improve and iterate!
Poor documentation can make the integration challenging
No documentation, or wrong documentation

How do you manage API usage and costs?

782 out of 782 answered

Use API keys with rate limits
388 resp.
49.6%
Evaluate API usage patterns to identify potential cost savings
300 resp.
38.4%
Use API analytics tools
175 resp.
22.4%
Implement billing and subscription models
168 resp.
21.5%
Consider using a cloud API management platform
100 resp.
12.8%

Many applications use data from multiple sources. Where do you combine data sources?

782 out of 782 answered

On the server-side
588 resp.
75.2%
On the client-side
319 resp.
40.8%
Syncing data sources to a single database
170 resp.
21.7%
A data federation service
72 resp.
9.2%
Other
14 resp.
1.8%
i dont
we have a single source, our own backend
Don't use multiple sources
In general, this does not happen at all, as a typical software landscape in a company consists of predominantly monolithic systems.
Any of the above depends on the application requirements
idk
Not sure
Never done that
not relevant
Data warehouse
N/A
N/A
N/A
"It depends"

How do you assess emerging trends and decide whether to learn more about them?

782 out of 782 answered

I monitor news, blogs, and social media to stay-up-to-date on emerging trends.
659 resp.
84.3%
I experiment with new technologies and trends on my own to see if they are a good fit for my needs.
476 resp.
60.9%
I rely on industry experts and thought leaders to identify emerging trends.
299 resp.
38.2%
I evaluate the potential benefits and risks of each emerging trend before deciding whether to adopt it.
293 resp.
37.5%
I attend conferences and workshops to learn about new technologies and trends.
172 resp.
22%
Other
8 resp.
1%
Trend is a very unspecific term here. Hardly anything has happened in the last 20 years that (somehow) wouldn't have been possible 20 years ago. People are simply learning to use technology better for themselves, but the basic principle remains the same. Replacing a gravel bed of black stones with white stones may look nicer to many, but the bottom line is that it remains a gravel bed.
Academic Research
I monitor news feeds and blogs to stay-up-to-date on emerging trends.
Dev friends
I just wait for the hype train to be over and see what survived it. Then check it out if i could need it.
idk
N/A
I judge by compatibility with current tools and technologies I am currently adopting.

What challenges do you encounter when adopting new data-related technologies or frameworks?

782 out of 782 answered

Integration with existing systems
507 resp.
64.8%
Learning curve for new technologies
506 resp.
64.7%
Assessing the impact on existing projects
363 resp.
46.4%
Concern about knowledge becoming outdated
263 resp.
33.6%
Other
24 resp.
3.1%
Longevity of the new technology or framework
Developer Experience
Sustainability
Lock in complexity - must be minimal or none at all
Assessing the impact on security.
Leadership enterprise level buyin
futureproofing
Tech/dependency obsolescence
Too much hype for technologies
Getting others on the team onboard and up to speed in a timely manner
Production readiness
Longevity/future-obsolescence of new tech
security concerns (SOC2, GDPR, HIPAA) and company resistance to using 3rd party SaaS products to product development
none, it's the same with old frameworks
Coworkers aren't always interested in taking the time to learn it for themselves
Fear of them becoming irrelevant soon
LTS (Long term support)
Having enough time to do everything
cross compatibility
whether new tech will be supported long-term
Data migration from one system to the next
Avoiding attractive dead-ends
Long term support
None

What emerging trend(s) in frontend/data handling do you believe will impact your work in the near future?

782 out of 782 answered

N/A
AI (for both helping the user with generating input as well as storing/analysing data)
AI
Databases as a service
Unsure…
move to cloud native
Don't know
New(but old) SSR-style architectures, like Astro. And low-code tools allowing the likes of myself (IT ops/repackager/scripter) actually build custom-DSL web apps (eg tracking App-packaging jobs in a multi-client team).)
Edge computing/storage
None in special
For now, I don't think any
?
React Server Components
HTMX
RSC
3d printers
AI
Phoenix LiveView, Phoenix LiveData
Modern data stack
Caching
RSC
AI and no code
Decentralized databases like GUN.Js
No one at this moment
Realtime + multiplayer

Are you using AI in any of the following capacities?

782 out of 782 answered

AI for code generation
502 resp.
64.2%
AI for recommendations
344 resp.
44%
AI for content creation
325 resp.
41.6%
AI for data analysis
198 resp.
25.3%
AI for decision-making
149 resp.
19.1%
AI for talk and process automation
133 resp.
17%
I'm not using AI
126 resp.
16.1%
Other
16 resp.
2%
I'm not using AI YET
Drafting solutions
AI for planning my chicken coop
As Documentation
ITSS ALL OVERATED, a useful tool, but just natural progression
AI for creating creative images that comes in my mind but don't know how to draw or create it physically or digitally. I know it comes under content creation but I think it's content creation is part of typing thing like text, speech etc...🫡
Im always using it to play around with ideas, most of the time i still end up writing everything on my own
Minimally for some low-impact awkward batch data processing.
Code Review and translations
for questions
AI for data retreival
AI for figuring out what I need to learn
assistance with exploring and learning new tech
Using it as a "better" Google / SO
Image gen
For review and general information

"AI is too risky for data analysis to influence decision-making."

782 out of 782 answered

2.6
Average rating

"AI is too risky to interact with a database and create, update, or delete records."

782 out of 782 answered

3.2
Average rating