
如有翻译问题欢迎评论指出,谢谢。
计算Pytorch模型参数量
-
Fábio Perez asked:
- 怎么计算 Pytorch 模型的参数量?类似 Keras 的
model.count_params()
那样的函数。
- 怎么计算 Pytorch 模型的参数量?类似 Keras 的
-
Answers:
-
Fábio Perez – vote: 198
-
Pytorch 没有类似 Keras 计算参数量的函数,但可以通过每个参数组的求和得出参数量:
-
pytorch_total_params = sum(p.numel() for p in model.parameters())
-
如果希望只计算可训练的参数:
-
pytorch_total_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
-
参考 Pytorch 论坛回答。
-
注:这个回答是自问自答,如果你们有更好的解决方法,请分享一下。
-
Fábio Perez – vote: 46
-
为了像 Keras 一样计算每层的参数量,Pytorch 的 model.named_paramters() 能返回一个迭代器,包含参数名与参数本身。
-
举例说明:
-
from prettytable import PrettyTable # def count_parameters(model): table = PrettyTable(["Modules", "Parameters"]) total_params = 0 for name, parameter in model.named_parameters(): if not parameter.requires_grad: continue params = parameter.numel() table.add_row([name, params]) total_params+=params print(table) print(f"Total Trainable Params: {total_params}") return total_params # count_parameters(net)
-
输出如下:
-
+-------------------+------------+ | Modules | Parameters | +-------------------+------------+ | embeddings.weight | 922866 | | conv1.weight | 1048576 | | conv1.bias | 1024 | | bn1.weight | 1024 | | bn1.bias | 1024 | | conv2.weight | 2097152 | | conv2.bias | 1024 | | bn2.weight | 1024 | | bn2.bias | 1024 | | conv3.weight | 2097152 | | conv3.bias | 1024 | | bn3.weight | 1024 | | bn3.bias | 1024 | | lin1.weight | 50331648 | | lin1.bias | 512 | | lin2.weight | 265728 | | lin2.bias | 519 | +-------------------+------------+ Total Trainable Params: 56773369
-
Thong Nguyen – vote: 12
-
如果希望避免重复计算共享的参数,可以用
torch.Tensor.data_ptr
,即: -
sum(dict((p.data_ptr(), p.numel()) for p in model.parameters()).values())
-
下面是更繁琐的实现,添加了一个选项,用于跳过不被训练的参数:
-
def numel(m: torch.nn.Module, only_trainable: bool = False): """ returns the total number of parameters used by
m
(only counting shared parameters once); ifonly_trainable
is True, then only includes parameters withrequires_grad = True
""" parameters = list(m.parameters()) if only_trainable: parameters = [p for p in parameters if p.requires_grad] unique = {p.data_ptr(): p for p in parameters}.values() return sum(p.numel() for p in unique)
-
Check the total number of parameters in a PyTorch model
-
Fábio Perez asked:
- How to count the total number of parameters in a PyTorch model? Something similar to
model.count_params()
in Keras.
怎么计算 Pytorch 模型的参数量?类似 Keras 的model.count_params()
那样的函数。
- How to count the total number of parameters in a PyTorch model? Something similar to
-
Answers:
-
Fábio Perez – vote: 198
-
PyTorch doesn\’t have a function to calculate the total number of parameters as Keras does, but it\’s possible to sum the number of elements for every parameter group:
Pytorch 没有类似 Keras 计算参数量的函数,但可以通过每个参数组的求和得出参数量: -
pytorch_total_params = sum(p.numel() for p in model.parameters())
-
If you want to calculate only the trainable parameters:
如果希望只计算可训练的参数: -
pytorch_total_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
-
Answer inspired by this answer on PyTorch Forums.
参考 Pytorch 论坛回答。 -
Note: I\’m answering my own question. If anyone has a better solution, please share with us.
注:这个回答是自问自答,如果你们有更好的解决方法,请分享一下。 -
Fábio Perez – vote: 46
-
To get the parameter count of each layer like Keras, PyTorch has model.named_paramters() that returns an iterator of both the parameter name and the parameter itself.
为了像 Keras 一样计算每层的参数量,Pytorch 的 model.named_paramters() 能返回一个迭代器,包含参数名与参数本身。 -
Here is an example:
举例说明: -
from prettytable import PrettyTable # def count_parameters(model): table = PrettyTable(["Modules", "Parameters"]) total_params = 0 for name, parameter in model.named_parameters(): if not parameter.requires_grad: continue params = parameter.numel() table.add_row([name, params]) total_params+=params print(table) print(f"Total Trainable Params: {total_params}") return total_params # count_parameters(net)
-
The output would look something like this:
输出如下: -
+-------------------+------------+ | Modules | Parameters | +-------------------+------------+ | embeddings.weight | 922866 | | conv1.weight | 1048576 | | conv1.bias | 1024 | | bn1.weight | 1024 | | bn1.bias | 1024 | | conv2.weight | 2097152 | | conv2.bias | 1024 | | bn2.weight | 1024 | | bn2.bias | 1024 | | conv3.weight | 2097152 | | conv3.bias | 1024 | | bn3.weight | 1024 | | bn3.bias | 1024 | | lin1.weight | 50331648 | | lin1.bias | 512 | | lin2.weight | 265728 | | lin2.bias | 519 | +-------------------+------------+ Total Trainable Params: 56773369
-
Thong Nguyen – vote: 12
-
If you want to avoid double counting shared parameters, you can use
torch.Tensor.data_ptr
. E.g.:
如果希望避免重复计算共享的参数,可以用torch.Tensor.data_ptr
,即: -
sum(dict((p.data_ptr(), p.numel()) for p in model.parameters()).values())
-
Here\’s a more verbose implementation that includes an option to filter out non-trainable parameters:
下面是更繁琐的实现,添加了一个选项,用于跳过不被训练的参数: -
def numel(m: torch.nn.Module, only_trainable: bool = False): """ returns the total number of parameters used by
m
(only counting shared parameters once); ifonly_trainable
is True, then only includes parameters withrequires_grad = True
""" parameters = list(m.parameters()) if only_trainable: parameters = [p for p in parameters if p.requires_grad] unique = {p.data_ptr(): p for p in parameters}.values() return sum(p.numel() for p in unique)
-
近期评论