TiDB Weekly

News & blog post

JD Cloud, owned by JD.com (China's largest online retailer), is a full-service cloud computing platform and integrated cloud service provider. Previously, they used MySQL to store OSS metadata. But as the metadata grew rapidly, standalone MySQL couldn't meet their storage requirements. They faced severe challenges in storing unprecedented amounts of data that kept soaring.

Last week, we published a post written by Can Cui, an infrastructure specialist at JD Cloud, that deep dives into how TiKV, an open-source distributed transactional key-value database, empowered JD Cloud to manage huge amounts of OSS metadata with a simple and horizontally scalable architecture. The post introduces why they chose TiKV, how they're using it, and some thoughts about future plans.

The full article is here:

Lesson Learned from 40 K QPS and 20+ Billion Rows of Data in a Single Scale-out Cluster

Community

New contributors

tidb:

• Tian-JQ
• gauss1314
• Thincher
• knock-C

tikv:

• Aloxaf
• Rustin-Liu
• hanahmily

pd:

• mantuliu

Call for participation

• Vectorized Expression Working Group

  • Date or Time built-in functions (63/65)
  • Decimal built-in functions (29/31)
  • Int built-in functions (165/187)
  • JSON built-in functions (23/27)
  • Real built-in functions (47/49)
  • String built-in functions (125/133)
  • Duration built-in functions (42/45)
  • Last week, 6 PRs were landed in the expression component.

• Coprocessor SIG:

  • Improve the performance of Chunk Encoder
  • Implement more UDFs
  • Implement the new row format
  • Port BinaryJSON
  • Tracing
  • The maximum execution time of Per-request

• Planner SIG:

  • Add rules for the cascades planner

• More issues to solve for TiDB

• More issues to solve for TiKV

Updates of the week

Progress

Last week, we landed 62 PRs in the TiDB repository, 20 PRs in the TiSpark repository, and 37 PRs in the TiKV and PD repositories.

Critical PRs

TiDB:

• Fix the bug that VIEWs can be dropped by the DROP TABLE syntax
• Fix a memory leak issue in batchClient for large transactions
• Add the MergeAggregationProjection transformation rule
• Support the index advisor and complete the input preparation part
• Make rowcodec used for mocktikv and unistore

TiSpark:

• Release TiSpark 2.1.8
• Fix the issue that Residual Filter contains wrong information
• Add TiChunkBatchColumnVector to the Chunk data in batches

TiKV and PD:

• Abstract a general forward scanner
• Support streaming for the diagnostics interface
• Add a flow control mechanism to ingest SST
• Add the maximum and minimum timestamp hint for the iterator
• Vectorize bit_count
• Vectorize any_value
• Vectorize rand and rand_with_seed

Upcoming TiDB event

Paper reading (in Chinese) on Dec 16