Column names and declared types are invariant for the lifetime of a
prepared statement, but Columns()/declTypes() called C.sqlite3_column_name
and sqlite3_column_decltype on every query, once per column. On hot
QueryRow paths reusing an explicit Prepare or a _stmt_cache_size statement,
this is a fixed set of cgo crossings paid on every execution.
Cache the names and decltypes on the SQLiteStmt the first time they are
materialized and reuse them on subsequent executions. Caching is gated by
cacheMetadata(): explicit prepared statements always cache, Query-created
ephemeral statements cache only when they live in the connection stmt
cache. One-shot statements keep the previous per-call behavior.
sqlite3_prepare_v2 returns SQLITE_OK with a NULL statement handle when
the input contains no SQL. exec() already handled this; query() forwarded
the NULL handle to bind(), which crashed in sqlite3_clear_bindings(NULL).
Make query() skip NULL statements like exec() does, and make SQLiteRows
safe against a nil underlying statement so the empty-rows return value
does not crash.
Closes#1390
This avoids an unnecessary reset when the cache is full, guarantees
a statement cannot enter the cache without being reset/cleared, and
fixes a leak where sqlite3_finalize was not called when reset failed.
Finalize all cached statements even if one fails. Leaving a
finalized statement in the cache map would be a use-after-finalize
bug per SQLite documentation.
Move extern declarations for _sqlite3_*_blocking functions before
_sqlite3_exec_no_args which references them. Remove unused
_sqlite3_prepare_v2_nolen function.
- Add _sqlite3_exec_no_args() C function that combines prepare+step+finalize
into a single CGO crossing for parameterless exec (most common case)
- Add _sqlite3_reset_clear() C function that combines sqlite3_reset and
sqlite3_clear_bindings into a single CGO crossing
- Use semaphore channel instead of result struct channel in context-aware
exec/Next paths to reduce allocations
- Use time.AppendFormat with stack buffer to avoid heap allocation in
time.Time binding
- Optimize bindNamedIndices to reuse a single buffer instead of 3
separate C.CString allocations
- Remove intermediate bindIndices slice allocation in named parameter
binding path
- Pass explicit query length to sqlite3_prepare_v2 to avoid C-side strlen
benchstat (n=8):
BenchmarkExec: -29.44% sec/op, -50% B/op, -33% allocs/op
BenchmarkQuery: -9.83% sec/op
BenchmarkParams: -6.38% sec/op
geomean: -6.72% sec/op
- Replace len(args[start:start+na]) > 0 with na > 0 to avoid slice bounds check
- Use range loops instead of manual index loops for cols/decltype slices
- Use range variable v.Ordinal instead of re-indexing args[i].Ordinal
- Add bounds hint for decltype access in nextSyncLocked loop
* Add ability to set an int64 file control
* Update documentation
* Remove duplicate err check in test
* Update sqlite3.go
Co-authored-by: rittneje <rittneje@gmail.com>
---------
Co-authored-by: rittneje <rittneje@gmail.com>
The commit removes the use of runtime.SetFinalizer to finalize
SQLiteRows since only serves to close the associated SQLiteStmt which
already has a registered finalizer.
It also fixes a race and potential panic in SQLiteRows.Close around the
SQLiteRows.s field (*SQLiteStmt) which is accessed without a mutex being
held, but modified with it held (null'd out). Further the mutex we are
holding is that of the SQLiteStmt so a subsequent call to Close will
cause a panic sine it'll attempt to dereference a nil field. The fix
here is to add a mutex for closing to SQLiteRows.
Since we now also set the s field to nil when closing this commit
removes the "closed" field (since checking if s is nil is the same) and
also changes the type of "nc" (number of columns) to an int32 so that we
can pack the nc and cls fields, and add the close mutex without making
the struct any bigger.
```
goos: darwin
goarch: arm64
pkg: github.com/charlievieth/go-sqlite3
cpu: Apple M4 Pro
│ x1.txt │ x4.txt │
│ sec/op │ sec/op vs base │
Suite/BenchmarkExec/Params-14 719.2n ± 2% 716.9n ± 1% ~ (p=0.897 n=10)
Suite/BenchmarkExec/NoParams-14 506.5n ± 3% 500.1n ± 0% -1.25% (p=0.002 n=10)
Suite/BenchmarkExecContext/Params-14 1.584µ ± 0% 1.567µ ± 1% -1.07% (p=0.007 n=10)
Suite/BenchmarkExecContext/NoParams-14 1.524µ ± 1% 1.524µ ± 1% ~ (p=0.539 n=10)
Suite/BenchmarkExecStep-14 443.9µ ± 3% 441.4µ ± 0% -0.55% (p=0.011 n=10)
Suite/BenchmarkExecContextStep-14 447.8µ ± 1% 442.9µ ± 0% -1.10% (p=0.000 n=10)
Suite/BenchmarkExecTx-14 1.643µ ± 1% 1.640µ ± 0% ~ (p=0.642 n=10)
Suite/BenchmarkQuery-14 1.968µ ± 3% 1.821µ ± 1% -7.52% (p=0.000 n=10)
Suite/BenchmarkQuerySimple-14 1.207µ ± 2% 1.040µ ± 1% -13.84% (p=0.000 n=10)
Suite/BenchmarkQueryContext/Background-14 2.400µ ± 1% 2.320µ ± 0% -3.31% (p=0.000 n=10)
Suite/BenchmarkQueryContext/WithCancel-14 8.847µ ± 5% 8.512µ ± 4% -3.79% (p=0.007 n=10)
Suite/BenchmarkParams-14 2.131µ ± 2% 1.967µ ± 1% -7.70% (p=0.000 n=10)
Suite/BenchmarkStmt-14 1.444µ ± 1% 1.359µ ± 1% -5.89% (p=0.000 n=10)
Suite/BenchmarkRows-14 61.57µ ± 1% 60.24µ ± 1% -2.16% (p=0.000 n=10)
Suite/BenchmarkStmtRows-14 60.15µ ± 1% 59.08µ ± 1% -1.78% (p=0.000 n=10)
Suite/BenchmarkQueryParallel-14 960.9n ± 1% 420.8n ± 2% -56.21% (p=0.000 n=10)
geomean 4.795µ 4.430µ -7.62%
```
This commit adds the SQLiteConn.FileControlInt() method which calls the
underlying sqlite3_file_control() function with an int argument. This can
be used for low-level operations on SQLite databases such as persisting
the WAL file after database close.
The busy_timeout pragma was added in sqlite 3.7.15 as an alternative
to calling sqlite3_busy_timeout directly:
https://sqlite.org/pragma.html#pragma_busy_timeout
While there's no functional change here, using the pragma does align
setting busy_timeout with other settings and removes the special case
for calling sqlite3_busy_timeout directly.
This can be used like in the test; I wrote a little wrapper around
sql.DB which uses this, and allows concurrent reads but just one single
write. This is perhaps a better generic "table locked"-solution than
setting the connections to 1 and/or cache=shared (although even better
would be to design your app in such a way that this doesn't happpen in
the first place, but even then a little seat belt isn't a bad thing).
The parsing adds about 0.1ms to 0.2ms of overhead in the wrapper, which
isn't too bad (and it caches the results, so only needs to do this
once).
At any rate, I can't really access functions from sqlite3-binding.c from
my application, so expose it via SQLiteStmt.
