Frank Liu, Director of Operations at Zilliz – Interview Collection


Frank Liu is the Director of Operations at Zilliz, a number one supplier of vector database and AI applied sciences. They’re additionally the engineers and scientists who created LF AI Milvus®, the world’s hottest open-source vector database.

What initially attracted you to machine studying?

My first publicity to the facility of ML/AI was as an undergrad scholar at Stanford, regardless of it being a bit far afield from my main (Electrical Engineering). I used to be initially drawn to EE as a subject as a result of the power to distill advanced electrical and bodily programs into mathematical approximations felt very highly effective to me, and statistics and machine studying felt the identical. I ended up taking extra pc imaginative and prescient and machine studying courses throughout grad faculty, and I ended up writing my Grasp’s thesis on utilizing ML to attain the aesthetic great thing about photographs. All of this led to my first job within the Pc Imaginative and prescient & Machine Studying workforce at Yahoo, the place I used to be in a hybrid analysis and software program growth function. We had been nonetheless within the pre-transformers AlexNet & VGG days again then, and seeing a complete subject and business transfer so quickly, from knowledge preparation to massively parallel mannequin coaching to mannequin productionization, has been wonderful. In some ways, it feels a bit ridiculous to make use of the phrase “again then” to discuss with one thing that occurred lower than 10 years in the past, however such is the progress that’s been made on this subject.

After Yahoo, I served because the CTO of a startup that I co-founded, the place we leveraged ML for indoor localization. There, we needed to optimize sequential fashions for very small microcontrollers – a really completely different however nonetheless associated engineering problem to at present’s huge LLMs and diffusion fashions. We additionally constructed {hardware}, dashboards for visualization, and easy cloud-native functions, however AI/ML all the time served as a core part of the work that we had been doing.

Though I’ve been in or adjoining to ML for the higher a part of 7 or 8 years now, I nonetheless preserve a number of love for circuit design and digital logic design. Having a background in Electrical Engineering is, in some ways, extremely useful for lots of the work that I’m concerned in lately as nicely. A number of essential ideas in digital design comparable to digital reminiscence, department prediction, and concurrent execution in HDL assist present a full-stack view to a number of ML and distributed programs at present. Whereas I perceive the attract of CS, I hope to see a resurgence in additional conventional engineering fields – EE, MechE, ChemE, and so forth… – inside the subsequent couple of years.

For readers who’re unfamiliar with the time period, what’s unstructured knowledge?

Unstructured knowledge refers to “advanced” knowledge, which is actually knowledge that can not be saved in a pre-defined format or match into an present knowledge mannequin. For comparability, structured knowledge refers to any kind of information that has a pre-defined construction – numeric knowledge, strings, tables, objects, and key/worth shops are all examples of structured knowledge.

To assist actually perceive what unstructured knowledge is and why it’s historically been troublesome to computationally course of this sort of knowledge, it helps to check it with structured knowledge. Within the easiest phrases, conventional structured knowledge might be saved through a relational mannequin. Take, for instance, a relational database with a desk for storing e-book info: every row inside the desk may signify a selected e-book listed by ISBN quantity, whereas the columns would denote the corresponding class of knowledge, comparable to title, creator, publish date, so on and so forth. These days, there are far more versatile knowledge fashions – wide-column shops, object databases, graph databases, so on and so forth. However the total concept stays the identical: these databases are supposed to retailer knowledge that matches a selected knowledge mould or knowledge mannequin.

Unstructured knowledge, however, might be regarded as primarily a pseudo-random blob of binary knowledge. It might signify something, be arbitrarily giant or small, and might be remodeled and browse in one in every of numerous alternative ways. This makes it not possible to suit into any knowledge mannequin, not to mention a desk in a relational database.

What are some examples of this sort of knowledge?

Human-generated knowledge – photographs, video, audio, pure language, and so forth – are nice examples of unstructured knowledge. However there are a selection of much less mundane examples of unstructured knowledge too. Consumer profiles, protein buildings, genome sequences, and even human-readable code are additionally nice examples of unstructured knowledge. The first purpose that unstructured knowledge has historically been so exhausting to handle is that unstructured knowledge can take any kind and might require vastly completely different runtimes to course of.

Utilizing photographs for example, two images of the identical scene may have vastly completely different pixel values, however each have the same total content material. Pure language is one other instance of unstructured knowledge that I wish to discuss with. The phrases “Electrical Engineering” and “Pc Science” are extraordinarily intently associated – a lot in order that the EE and CS buildings at Stanford are adjoining to one another – however with out a technique to encode the semantic which means behind these two phrases, a pc could naively suppose that “Pc Science” and “Social Science” are extra associated.

What’s a vector database?

To grasp a vector database, it first helps to grasp what an embedding is. I’ll get to that momentarily, however the brief model is that an embedding is a high-dimensional vector that may signify the semantics of unstructured knowledge. Typically, two embeddings that are shut to at least one one other when it comes to distance are very more likely to correspond to semantically related enter knowledge. With fashionable ML, we have now the facility to encode and remodel quite a lot of several types of unstructured knowledge – photographs and textual content, for instance – into semantically highly effective embedding vectors.

From a company’s perspective, unstructured knowledge turns into extremely troublesome to handle as soon as the quantity grows previous a sure restrict. That is the place a vector database comparable to Zilliz Cloud is available in. A vector database is purpose-built to retailer, index, and search throughout huge portions of unstructured knowledge by leveraging embeddings because the underlying illustration. Looking throughout a vector database is often achieved with question vectors, and the results of the question is the highest N most related outcomes based mostly on distance.

The perfect vector databases have lots of the usability options of conventional relational databases: horizontal scaling, caching, replication, failover, and question execution are simply among the many options {that a} true vector database ought to implement. As a class definer, we’ve been lively in educational circles as nicely, having printed papers in SIGMOD 2021 and VLDB 2022, the 2 high database conferences on the market at present.

Might you focus on what an embedding is?

Typically talking, an embedding is a high-dimensional vector that comes from the activations of an intermediate layer in a multilayer neural community. Many neural networks are skilled to output embeddings themselves and a few functions use concatenated vectors from a number of intermediate layers because the embedding, however I received’t get too deep into both of these for now. One other much less frequent however equally essential technique to generate embeddings is thru handcrafted options. Somewhat than having an ML mannequin mechanically study the fitting representations for the enter knowledge, good previous function engineering can work for a lot of functions as nicely. Whatever the underlying methodology, embeddings for semantically related objects are shut to one another when it comes to distance, and this property is what powers vector databases.

What are among the hottest use instances with this know-how?

Vector databases are nice for any software that requires some type of semantic search – product advice, video evaluation, doc search, risk & fraud detection, and AI-powered chatbots are among the hottest use instances for vector databases at present. For example this, Milvus, the open-source vector database created by Zilliz and the underlying core of Zilliz Cloud, has been utilized by over a thousand enterprise customers throughout quite a lot of completely different use instances.

I’m all the time pleased to speak about these functions and assist of us perceive how they work, however I undoubtedly tremendously take pleasure in going over among the lesser-known vector database use instances as nicely. New drug discovery is one in every of my favourite “area of interest” vector database use instances. The problem for this specific software is trying to find potential candidate medication to deal with a sure illness or symptom amongst a database of 800 million compounds. A pharmaceutical firm we communicated with was capable of considerably enhance the drug discovery course of along with reducing down on {hardware} assets by combining Milvus with a cheminformatics library referred to as RDKit.

Cleveland Museum of Artwork’s (CMA) AI ArtLens is one other instance I wish to carry up. AI ArtLens is an interactive device that takes a question picture as an enter and pulls visually related photographs from the museum’s database. That is normally known as reverse picture search and is a reasonably frequent use case for vector databases, however the distinctive worth proposition that Milvus supplied to CMA was the power to get the applying up and working inside every week with a really small workforce.

Might you focus on what the open-source platform Towhee is?

When speaking with of us from the Milvus neighborhood, we discovered that a lot of them wished to have a unified technique to generate embeddings for Milvus. This was true for almost all the completely different organizations that we spoke with, however particularly so for corporations that didn’t have many machine studying engineers. With Towhee, we intention to unravel this hole through what we name “vector knowledge ETL.” Whereas conventional ETL pipelines concentrate on combining and remodeling structured knowledge from a number of sources right into a usable format, Towhee is supposed to work with unstructured knowledge and explicitly contains ML within the ensuing ETL pipeline. Towhee accomplishes this by offering a whole bunch of fashions, algorithms, and transformations that can be utilized as constructing blocks in a vector knowledge ETL pipeline. On high of this, Towhee additionally supplies an easy-to-use Python API which permits builders to construct and take a look at these ETL pipelines in a single line of code.

Whereas Towhee is its personal impartial undertaking, additionally it is part of the broader vector database ecosystem centered round Milvus that Zilliz is creating. We envision Milvus and Towhee to be two extremely complementary initiatives which, when used collectively, can actually democratize unstructured knowledge processing.

Zilliz just lately raised a $60M Collection B spherical. How will this speed up the Zilliz mission?

I’d first off wish to thank Prosperity7 Ventures, Pavilion Capital, Hillhouse Capital, 5Y Capital, Yunqi Capital, and others for believing in Zilliz’s mission and supporting us with this Collection B extension. We’ve now raised a complete of $113M, and this newest spherical of funding will assist our efforts to scale out engineering and go-to-market groups. Particularly, we’ll be bettering our managed cloud providing, which is at the moment in early entry however scheduled to divulge heart’s contents to everyone later this yr. We’ll additionally proceed to put money into cutting-edge database & AI analysis as we have now achieved previously 4 years.

Is there the rest that you simply wish to share about Zilliz?

As an organization, we’re rising quickly, however what actually units our present workforce other than others within the database and ML house is our singular ardour for what we’re constructing. We’re on a mission to democratize unstructured knowledge processing, and it’s completely wonderful to see so many gifted of us at Zilliz working in the direction of a singular objective. If any of what we’re doing sounds attention-grabbing to you, be at liberty to get in contact with us. We’d like to have you ever onboard.

In case you’d wish to know a bit extra, I’m additionally personally open to chatting about Zilliz, vector databases, or embedding-related developments in AI/ML. My (figurative) door is all the time open, so be at liberty to succeed in out to me immediately on Twitter/LinkedIn.

Final however not least, thanks for studying!

Thanks for the good interview, readers who want to study extra ought to go to Zilliz.

Leave a Reply