Posts

Here’s a pretty cool article on understanding PyTorch conv1d shapes for text classification. In this article, the shape of the example is: n = 1: number of batches d = 3: dimension of the word embedding l = 5: length of the sentence import torch.nn as nn import torch # Example represents one sentence here example = torch.rand(n=1, l=3, d=5) example.shape # torch.Size([1, 5, 3]) example # This is the output: tensor([[[0.0959, 0.1674, 0.1259], [0.8330, 0.5789, 0.2141], [0.3774, 0.8055, 0.4218], [0.1992, 0.4722, 0.3167], [0.4633, 0.0352, 0.8803]]]) In the above output, you can image each row represents one word. ...

Introduction In the XCS330 Problem Set 4, we will explore methods for performing few-shot learning with pre-trained LMs. Datasets from HuggingFace: Amazon Reviews: five-way classification problem. Given a review, predict the start rating, from 1 to 5. XSum: news articles from BBC. Given a news, return a one sentence summary. Use n-gram overlap to measure the score bAbI: question-answering tasks that requires reasoning about contextual information. AI benchmark developed by Facebook AI Research. Fine-tuning Here, we fine tune the entire model on k examples using two diffrent sizes of samller BERT models. ...

Introduction In traditional machine learning, we have a lot of dataset for a specific tasks, while in meta-learning, we have many tasks with small datasets, and the hope is that we can train a model that can learn some fundamental idea from other tasks. Put it this way: the goal of traditional ML is to optimize the performance of a single task, while the goal of meta-learning is to optimize for adaptability. ...

Softmax The softmax function is widely used in machine learning. It is used to convert a vector of raw values into a probability distribution. Given a vector $\mathbf{z} = [z_1, z_2, \ldots, z_n]$, the softmax function is defined as: $$ \sigma(\mathbf{z_i}) = \frac{e^{z_i}}{\sum_{j=1}^{n} e^{z_j}} $$ Here’s an example in PyTorch: import torch import torch.nn.functional as F logits = logits = torch.tensor([5, 4, -1], dtype=float) prbabilities = F.softmax(logits) print(probabilities) # tensor([0.7297, 0.2685, 0.0018], dtype=torch.float64)

Install Conda Since I am a mac user and I want to leverage the Apple Silicon CPU architecture, I installed Miniforge. During installation, I chose not to have conda to control my shell environment, and therefore after installation, I need to run the following command to activate the base environment. source ~/miniforge3/bin/activate Windows Users For windows users, use this link. After installation, search in the start menu with Anacoda Prompt. You should be able to see a special cmd prompt with the Anaconda icon in the bottom right corner. ...

Latex tutorial Greek numbers, for instance \textlambda. https://www.overleaf.com/learn/latex/List_of_Greek_letters_and_math_symbols Superscript: ^ and subscript: _, example: $a_i$ Basic bold text: https://www.overleaf.com/learn/latex/Bold%2C_italics_and_underlining \textbf{} Greek Letters Greek Letters and Math Symbols. Table How to write a table in Latex: https://www.overleaf.com/learn/latex/Tables A little complicated. \begin{center} \begin{tabular}{ |c|c|c| } \hline cell1 & cell2 & cell3 \\ cell4 & cell5 & cell6 \\ cell7 & cell8 & cell9 \\ \hline \end{tabular} \end{center} Linux Commands This is how to add Linux commands: ...

In the 3rd assignment of XCS330, we will implement prototypical neworks (protonets) for few-shots image classification on the Omniglot dataset. Protonets Algorithm This is the protonet in a nutshell, and the example comes from the assignment: In this example, we compute three class prototypes c1, c2, c3 from the support features. The decision boundaries are computed using Euclidean distance. When there’s a new query, we can determine which class it belongs to. ...

In this blog post, I will share the findings of my research in the field of threat modeling. What Is Threat Modeling Threat modeling is a process used to identify, assess, and address potential security threats and vulnerabilities in a system. The STRIDE Model There are numerous threat modeling methodologies, and the one I am interested in is the STRIDE model: Category Description Spoofing Impersonating another user or device. For example, IP spoofing: altering the source IP address in a packet header to make it appear as though the packet is coming from a trusted source. Tampering Malicious modification of data. For example, altering the data as it flows between two computers over the internet. Repudiation User denies having performed an action. For example, a user denies making a billion dolar transaction, and the system lacks the evidence to prove the transaction was indeed made by the user. Information Disclosure exposure of information to individuals who are not supposed to have access to it. For example, users read a file that they were not granted access to. Denial of service Denial of service (DoS) attack Elevation of Privilege An unprivileged user gains privileged access and thereby has sufficient access to compromise or destroy the entire system. Tooling There are many toolings for threat modeling and here are some of them: ...

Introduction In this blog, I want to talk about how to leverage az cli to visualize a graph in Python. Note that Azure portal has a tab that shows the visualization of the resources. The reason why I am still interested in this topic is that I want to turn azure resource definition into a data structure, which can then be leveraged to generate different form of diagrams: dependency graph of azure resources data flow diagram (for threat modeling) mermaid diagram (for documentation) sequence diagram (need to get sequence information from users) There are also other tools for visualizing azure resources: ...

Intro SQLAlchemy is a very popular ORM Python library. Key Concepts Engine See Engine Configuration. The engine is the starting point for SQLAlchemy application. The very basic usage is to create an engine from a postgresql url: from sqlalchemy import create_engine engine = create_engine("postgresql+psycopg2://scott:tiger@localhost:5432/mydatabase") Session See Session Basics. A session is a “holding zone” for the orm-mapped objects and the database. we can configure properties like autocommit and autoflush, and these settings can impact the performance of the application. ...